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Preface 



These are the proceedings of the 8tlr International Workshop on Cooperative 
Information Agents (CIA 2004), held at the Fair and Congress Center in Er- 
furt, Germany, September 27 -29, 2004. It was part of the multi-conference 
Net.ObjectDays 2004, and, in particular, was co-located with the 2nd German 
Conference on Multiagent Systems Technologies (MATES 2004). 

In today’s networked world of linked heterogeneous, pervasive computer 
systems, devices, and information landscapes, the intelligent coordination and 
provision of relevant added- value information at any time, anywhere, by means 
of cooperative information agents becomes increasingly important for a variety 
of applications. An information agent is a computational software entity that 
has access to one or multiple, heterogeneous, and geographically dispersed data 
and information sources. It proactively searches for and maintains information 
on behalf of its human users, or other agents, preferably just in time. In other 
words, it is managing and overcoming the difficulties associated with information 
overload in open, pervasive information and service landscapes. Cooperative in- 
formation agents may collaborate with each other to accomplish both individual 
and shared joint goals depending on the actual preferences of their users, bud- 
getary constraints, and resources available. One major challenge of developing 
agent-based intelligent information systems in open environments is to balance 
the autonomy of networked data, information, and knowledge sources with the 
potential payoff of leveraging them using information agents. 

Interdisciplinary research and development of information agents requires ex- 
pertise in relevant domains of information retrieval, artificial intelligence, 
database systems, human-computer interaction, and Internet and Web technol- 
ogy. The CIA 2004 workshop aimed at providing an interdisciplinary forum for 
researchers, software developers, and managers to get informed about, present, 
and discuss the latest high-quality results in advancements of theory and prac- 
tice in information agent technology for the Internet, the Web, and the Semantic 
Web. 

The CIA workshop series was initiated in 1997; since then the events of the 
series have been held annually, mainly at different locations in Europe. Each 
event of the series offers regular and invited talks of excellence that are given 
by distinguished experts, and system demonstrations, and honors innovative re- 
search and development on information agents by means of best paper awards 
and system innovation awards. The proceedings of the series are regularly pub- 
lished as volumes of the Lecture Notes in Artificial Intelligence (LNAI) series of 
Springer. 

In keeping with its tradition, this year’s workshop featured a sequence of 
regular and invited talks of excellence given by leading experts in the field. These 
talks covered a broad area of topics of interest, such as information agents for 
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pervasive and peer-to-peer computing environments; issues of communication 
and cooperation; industrial applications; and recommender agents. 

This year the CIA System Innovation Award and the CIA Best Paper Award 
were sponsored by Whitestein Technologies AG, Switzerland, and the Spanish 
Association for Artificial Intelligence (AEPIA), respectively. The DMR Decision 
Engineering Lab of the Universidad Rey Juan Carlos in Madrid, Spain provided 
limited financial support to students who were co-authors of accepted papers so 
that they could give their presentations at the CIA 2004 workshop. 

CIA 2004 featured 3 invited and 18 regular papers selected from 59 submis- 
sions. The result of the peer-review of all contributions is included in this volume, 
rich with interesting, inspiring, and advanced work on research and development 
on intelligent information agents worldwide. All previous workshop proceedings 
were published by Springer as Lecture Notes in Artificial Intelligence volumes: 
1202 (1997), 1435 (1998), 1652 (1999), 1860 (2000), 2182 (2001), 2446 (2002), 
and 2782 (2003). 

The CIA 2004 workshop was organized in cooperation with the Association 
for Computing Machinery (ACM). In addition, we are very much indebted to 
our sponsors, whose financial support made this event possible. The sponsors of 
CIA 2004 were: 



TRANSIT GmbH, Germany 
URJC Decision Engineering Lab (DMR), Spain 
Whitestein Technologies, Switzerland 
Spanish Association for Artificial Intelligence, Spain 
AgentLink III, EU FP6 Coordinating Action 

We are in particular grateful to the authors and invited speakers for con- 
tributing their latest and inspiring work to this workshop, as well as to the 
members of the program committee, and the external reviewers for their critical 
reviews of submissions. Finally, a deep thanks goes to each of the brave members 
of the local organization team from tranSIT GmbH at the Fair and Congress 
Center in Erfurt for their hard work in providing the CIA 2004 event with a 
modern, comfortable location, and a social program. 



September 2004 



Matthias Klusch, Sasclra Ossowski, 
Vipul Kashyap, Rainer Unland 
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Abstract. Multi-agent systems offer an architecture for distributed problem solv- 
ing. Distributed data mining algorithms specialize on one class of such distributed 
problem solving tasks — analysis and modeling of distributed data. This paper of- 
fers a perspective on distributed data mining algorithms in the context of multi- 
agents systems. It particularly focuses on distributed clustering algorithms and 
their potential applications in multi-agent-based problem solving scenarios. It dis- 
cusses potential applications in the sensor network domain, reviews some of the 
existing techniques, and identifies future possibilities in combining multi-agent 
systems with the distributed data mining technology. 

Keywords: multi-agent systems, distributed data mining, clustering 



1 Introduction 

Multi-agent systems (MAS) often deal with complex applications that require distributed 
problem solving. In many applications the individual and collective behavior of the 
agents depend on the observed data from distributed sources. In a typical distributed 
environment analyzing distributed data is a non-trivial problem because of many con- 
straints such as limited bandwidth (e.g. wireless networks), privacy-sensitive data, dis- 
tributed compute nodes, only to mention a few. The field of Distributed Data Mining 
(DDM) deals with these challenges in analyzing distributed data and offers many al- 
gorithmic solutions to perform different data analysis and mining operations in a fun- 
damentally distributed manner that pays careful attention to the resource constraints. 
Since multi-agent systems are also distributed systems, combining DDM with MAS for 
data intensive applications is appealing. 

This paper makes an effort to underscore the possible synergy between multi-agent 
systems and distributed data mining technology. It particularly focuses on distributed 
clustering, a problem finding increasing number of applications in sensor networks, 
distributed information retrieval, and many other domains. The paper discusses one of 
these application domains, illustrates the ideas, and reviews existing work in this area. 
Although, the power of DDM is not just restricted to clustering, this paper chooses to 
restrict the scope for the sake of brevity. 

The paper is organized as follows. Section 2 provides the motivation behind the 
material presented in this paper. Section 3 introduces DDM and presents an overview 
of the field. Section 4 focuses on a particular portion of the DDM literature and takes 
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an in-depth look at the distributed clustering literature. Section 5 considers distributed 
clustering algorithms in the context of sensor networks that are drawing an increasing 
amount of interest from the multi-agent systems community. Finally, Section 6 con- 
cludes this paper. 



2 Motivation 

Agents in MAS need to be pro-active and autonomous. Agents perceive their envi- 
ronment, dynamically reason out actions based on conditions, and interact with each 
other. In some applications the knowledge of the agents that guide reasoning and ac- 
tion depend on the existing domain theory. However, in many complex domains this 
knowledge is a result of existing domain theory and also the outcome of empirical data 
analysis. Scalable analysis of data may require advanced data mining for detecting hid- 
den patterns, constructing predictive models, and identifying outliers, among others. In 
a multi-agent system this knowledge is usually collective. This collective “intelligence” 
of a multi-agent system must be developed by distributed domain knowledge and anal- 
ysis of distributed data observed by different agents. Such distributed data analysis may 
be a non-trivial problem when the underlying task is not completely decomposable and 
computing resources are constrained by several factors such as limited power supply, 
poor bandwidth connection, and privacy sensitive multi-party data, among others. 

For example, consider a defense related application of monitoring a terrain using 
a sensor network that has many tiny mote-type [15] sensors for measuring vibration, 
reflectance, temperature, and audio signals. Let us say the objective is to identify and 
track a certain type of vehicle ( e.g . pick-up trucks). The sensors are battery-powered. 
Therefore, in the normal mode they are designed not be very active. However, as soon as 
someone detects a possible change in scenario, the sensors must wake up, observe, rea- 
son, and collaborate with each other in order to track and identify the object of interest. 
The observations are usually time-series data sampled at a device specific rate. There- 
fore, collaboration with other sensor-nodes would require comparing data observed at 
different nodes. This usually requires sending a window of observations from one node 
to another node. This distributed problem solving environment appears to fit very well 
with the multi-agent framework since the solution requires semi-autonomous behavior, 
collaboration and reasoning among other things. However, regardless of how sophisti- 
cated the agents are, from the domain knowledge and reasoning perspective, they must 
perform the underlying data analysis tasks very efficiently in a distributed manner. The 
traditional framework of centralized data analysis and mining algorithms does not really 
scale very well in such distributed applications. For example, if we want to compare the 
data vectors observed at different sensor nodes the centralized approach will be to send 
the data vectors to the base station (usually connected through a wireless network) and 
then compare the vectors using whatever metric is appropriate for the domain. This does 
not scale up in large sensor networks since data transmission consumes a lot of battery 
power and heavy data transmission over limited bandwidth channel may produce poor 
response time. Distributed data mining technology offers more efficient solutions in 
such applications. The following discussion illustrates the power of DDM algorithms us- 
ing a simple randomized technique for addressing this sensor network-related problem. 
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Given vectors a = (ai, . . . , a m ) T and b = (bi , . . . , b m ) T at two distributed site 
A and B, respectively, we want to approximate the Euclidean distance between them 
using a small number (compared to to) of messages between A and B. Note that the 
problem of computing the Euclidean distance between a pair of data tuples a and b 
can be represented as the problem of computing the inner products between them as 
follows: 



dg(o, b) =< a, a > + < b, b > — 2 < a, b > 

where d%(a, b) denotes the Euclidean distance between a and b: < a. b > represents 
the inner product between a and b, defined as i a ih- Therefore, the core challenge 
is to develop an algorithm for distributed inner product computation. One can approach 
this problem in several ways. Table 1 shows a simple communication efficient random- 
ized technique for computing the inner product between two vectors observed at two 
different sites. 



Algorithm 2.0.1 Distributed Dot Product Algorith m(a, b) 

1. A sends B a random number generator seed. [1 message] 

2. A and B cooperatively generate k x to random matrix R where k <C to. Each entry is gen- 
erated independently and identically from some fixed distribution with mean zero and variance 
one. A and B compute a = Ra, b = Rb, respectively. 

3. A sends a to B. B computes a T b = a T R T Rb. [k messages] 

4. B computes D = 



So instead of sending a m-dimensional vector to the other site, we only need to 
send a /c-dimensional vector where k to and the dot product can still be estimated 
accurately. Indeed, it can be shown that the expected value of D is < a, b > and Table 1 
shows some experimental results concerning accuracy. 

This algorithm illustrates a simple communication-efficient-way to compare a pair 
of data vectors observed at two different nodes. It potentially offers a building block to 
support the collaborative object identification and tracking problem in sensor networks 
where the centralized solution does not work because of limited bandwidth and power 
supply for the sensor nodes. 

Privacy of the data can be another reason for adopting the DDM technology. In many 
applications, particularly in security-related applications, data are privacy-sensitive. 
When the data are multi-party and privacy-sensitive, centralizing the data is usually 
not acceptable. Therefore, many data mining applications in such domains must ana- 
lyze data in a distributed fashion without having to first download everything to a single 
site. There exists a growing number of DDM algorithms that address many data mining 
problems for distributed environments. The following section presents an overview. 



3 Distributed Data Mining: A Brief Overview 

Data mining [9] , [ 1 0] , [ 1 1 ] ,and [31] deals with the problem of analyzing data in scalable 
manner. Distributed data mining is a branch of the field of data mining that offers a 
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Table 1 . Relative errors in computing the dot product between two synthetic binary vectors each 
with 10000 elements, k is the number of randomized iterations, k is also represented as the per- 
centage of the size of the original vectors. Each entry of the random matrix is chosen indepen- 
dently from U(l, — 1 ) 



k 


Mean 


Var 


Min 


Max 


100(1%) 


0.1483 


0.0098 


0.0042 


0.3837 


500(5%) 


0.0795 


0.0035 


0.0067 


0.2686 


1000(10%) 


0.0430 


0.0008 


0.0033 


0.1357 


2000(20%) 


0.0299 


0.0007 


0.0012 


0.0902 


3000(30%) 


0.0262 


0.0005 


0.0002 


0.0732 



framework to mine distributed data paying careful attention to the distributed data and 
computing resources. 

In the DDM literature, one of two assumptions is commonly adopted as to how 
data is distributed across sites: homogeneously (horizontally partitioned) and heteroge- 
neously (vertically partitioned). Both viewpoints adopt the conceptual viewpoint that 
the data tables at each site are partitions of a single global table. In the homogeneous 
case, the global table is horizontally partitioned. The tables at each site are subsets of the 
global table; they have exactly the same attributes. In the heterogeneous case the table 
is vertically partitioned, each site contains a collection of columns (sites do not have 
the same attributes). However, each tuple at each site is assumed to contain a unique 
identifier to facilitate matching. It is important to stress that the global table viewpoint 
is strictly conceptual. It is not necessarily assumed that such a table was physically 
realized and partitioned to form the tables at each site. Figures 1 and 2 illustrate the ho- 
mogeneously distributed case using an example from weather data. Both tables use the 
same set of attributes. On the other hand, Figures 3 and 4 illustrate the heterogeneously 
distributed case. The tables have different attributes and tuples are linked through a 
unique identifier. Timestamp. 

The development of data mining algorithms that work well under the constraints 
imposed by distributed datasets has received significant attention from the data min- 
ing community in recent years. The field of DDM has emerged as an active area of 
study. The bulk of DDM methods in the literature operate over an abstract architec- 
ture which includes multiple sites having independent computing power and storage 
capability. Local computation is done on each of the sites and either a central site com- 
municates with each distributed site to compute the global models or a peer-to-peer 
architecture is used. In the latter case, individual nodes might communicate with a re- 
source rich centralized node, but they perform most of the tasks by communicating with 
neighboring nodes by message passing over an asynchronous network. For example, the 
sites may represent independent sensor nodes which connect to each other in an ad-hoc 
fashion. 

Some features of a distributed scenario where DDM is applicable are as follows. 

1. The system consist of multiple independent sites of data and computation which 
communicate only through message passing. 

2. Communication between the sites is expensive. 
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3. Sites may have resource constraints. 

4. Sites may have privacy concerns. 

Typically communication is a bottleneck. Since communication is assumed to be 
carried out exclusively by message passing, a primary goal of many DDM methods in 
the literature is to minimize the number of messages sent. Some methods also attempt 
to load-balance across sites to prevent performance from being dominated by the time 
and space usage of any individual site. As pointed out in [25], "Building a monolithic 
database, in order to perform non-distributed data mining, may be infeasible or simply 
impossible’’ in many applications. The cost of transferring large blocks of data may be 
prohibitive and result in very inefficient implementations. 

Surveys [17] and [24] provide a broad, up-to-date overview of DDM touching on is- 
sues such as: clustering, association rule mining, basic statistics computation, Bayesian 
network learning, classification, the historical roots of DDM. The collection [16] de- 
scribes a variety of DDM algorithms (association rule mining, clustering, classification, 
preprocessing, etc.), systems issues in DDM (security, architecture, etc.), and some top- 
ics in parallel data mining. Survey [33] discusses parallel and distributed association 
rule mining in DDM. Survey [34] discusses a broad spectrum of issues in DDM and 
parallel data mining and provides a survey of distributed and parallel association rule 
mining and clustering. Many of the DDM applications [27, 18] deal with continuous 
data streams. Therefore, developing DDM algorithms that can handle such stream sce- 
narios is becoming increasingly important. An overview of the data stream mining lit- 
erature can be found elsewhere [2], 

Instead of looking at the broad spectrum of different DDM algorithms, this paper 
restricts itself to distributed clustering methods and their applicability in multi-agent 
systems. The following section addresses this issue. 



City 


Humidity 


Temperature 


Rainfall 


Baltimore 


10% 


23° F 


Oin. 


Annapolis 


13% 


43° F 


0.2 in. 


Bethesda 


56% 


67° F 


1 in. 


Glen Burnie 


88% 


88° F 


1.2 in. 



Fig. 1. Homogeneously distributed weather data at site 1 
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Humidity 


Temperature 


Rainfall 


San Jose 


12% 


69° F 


0.3 in. 


Sacramento 


18% 


53° F 


0.5 in. 


Los Angeles 


86% 


72° F 


1.2 in. 


San Diego 


8% 


58° F 


Oin. 



Fig. 2. Homogeneously distributed weather data at site 2 
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Timestamp 


Humidity 


Temperature 


Rainfall 


to 


10% 


23° F 


Oin. 


tl 


13% 


43° F 


0.2 in. 


t2 


56% 


67° F 


1 in. 


f3 


88% 


88° F 


1.2 in. 



Fig. 3. Heterogeneously distributed weather data 
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to 
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tl 


1.9 quart 


7.8 


12 quart 


t2 


2.7 quart 


6.7 


13 quart 


t3 


4.2 quart 


5.2 


14.5 quart 



Fig. 4. Heterogeneously distributed grocery store data 



4 Distributed Clustering Algorithms 

In this section, we present an overview of various distributed clustering solutions pro- 
posed to date. We classify distributed clustering algorithms into two categories. The 
first group consists of methods requiring multiple rounds of message passing. These 
methods require a significant amount synchronization. The second group consists of 
methods that build local clustering models and transmit them to a central site (asyn- 
chronously). The central site forms a combined global model. These methods require 
only a single round of message passing, hence, modest synchronization requirements. 

4.1 Multiple Communication Round Algorithms 

Dhillon and Modha [3] develop a parallel implementation of the A'-means clustering al- 
gorithm on distributed memory multiprocessors (homogeneously distributed data). The 
algorithm makes use of the inherent data parallelism in the A'-means algorithm. Given 
a dataset of size n, they divide it into P blocks, (each of size roughly n/P). During 
each iteration of A'-means, each site computes an update of the current I\ centroids 
based on its own data. The sites broadcast their centroids. Once a site has received all 
the centroids from other sites it can form the global centroids by averaging. 

Forman and Zhang [8] take an approach similar to the one presented in [3], but 
extend it to A'-harmonic means. Note that the methods of [3] and [8] both start by par- 
titioning and then distributing a centralized data set over many sites. This is different 
than the setting we consider: the data is never centralized - it is inherently distributed. 
However, their ideas are useful for designing algorithms to cluster homogeneously dis- 
tributed data. 

Kargupta et al. [19] develop a collective principle components analysis (PCA)- 
based clustering technique for heterogeneously distributed data. Each local site per- 
forms PCA, projects the local data along the principle components, and applies a known 
clustering algorithm. Having obtained these local clusters, each site sends a small set 
of representative data points to a central site. This site carries out PCA on this collected 
data (computes global principal components). The global principle components are sent 
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back to the local sites. Each site projects its data along the global principle components 
and applies its clustering algorithm. A description of locally constructed clusters is sent 
to the central site which combines the cluster descriptions using different techniques 
including but not limited to nearest neighbor methods. 

Klusch et al. [20] consider kernel-density based clustering over homogeneously dis- 
tributed data. They adopt the definition of a density based cluster from [12] data points 
which can be connected by an uphill path to a local maxima, with respect to the kernel 
density function over the whole dataset, are deemed to be in the same cluster. Their 
algorithm does not find a clustering of the entire dataset. Instead each local site finds a 
clustering of its local data based on the kernel density function computed over all the 
data. In principle, their approach could be extended to produce a global clustering by 
transmitting the local clusterings to a central site and then combining them. However, 
carrying out this extension in a communication efficient manner is non-trivial task and 
is not discussed by Klusch et al. 

An approximation to the global, kernel density function is computed at each site 
using sampling theory from signal processing. The sites must first agree upon a cube 
and a grid (of the cube). Each corner point can be thought of as a sample from the space 
(not the data set). Then each site computes the value of its local density function at each 
corner of the grid and transmits the corner points along with their local density values 
to a central site. The central site computes the sum of all samples at each grid point 
and transmits the combined sample grid back to each site. The local sites can now inde- 
pendently estimate the global density function over all points in the cube (not just the 
corner points) using techniques from sampling theory in signal processing. The local 
sites independently apply a gradient-ascent based density clustering algorithm to arrive 
at a clustering of their local data. 

Eisenhardt et al. [5] develop a distributed method for document clustering (hence 
operates on homogeneously distributed data). They extend A -means with a “probe and 
echo’’ mechanism for updating cluster centroids. Each synchronization round corre- 
sponds to a K -means iteration. Each site carries out the following algorithm at each it- 
eration. One site initiates the process by marking itself as engaged and sending a probe 
message to all its neighbors. The message also contains the cluster centroids currently 
maintained at the initiator site. The first time a node receives a probe (from a neighbor 
site p with centroids C v ), it marks itself as engaged, sends a probe message (along with 
C p ) to all its neighbors (except the origin of the probe), and updates the centroids in 
C p using its local data as well as computing a weight for each centroid based on the 
number of data points associated with each. If a site receives an echo from a neighbor 
p (with centroids C p and weights W p ), it merges C p and W p with its current centroids 
and weights. Once a site has received either a probe or echo from all neighbors, it sends 
an echo along with its local centroids and weights to the neighbor from which it re- 
ceived its first probe. When the initiator has received echos from all its neighbors, it has 
the centroids and weights which take into account all datasets at all sites. The iteration 
terminates. 

While all algorithms in this section require multiple rounds of message passing, [19] 
and [20] require only two rounds. The others require as many rounds as the algorithm 
iterates (potentially many more than two). 
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4.2 Centralized Ensemble-Based Methods 

Many of the distributed clustering algorithms work in an asynchronous manner by first 
generating the local clusters and then combining those at the central site. These ap- 
proaches potentially offer two nice properties in addition to lower synchronization re- 
quirements. If the local models are much smaller than the local data, their transmission 
will result is excellent message load requirements. Moreover, sharing only the local 
models may be a reasonable solution to privacy constraints in some situations; indeed, 
a trade-off between privacy and communication cost is discussed in [22]. 

We present the literature in chronological order. Some of the methods were not 
explicitly developed for distributed clustering, rather for combining clusterings in a 
centralized setting to produce a better overall clustering. In these cases we discuss how 
well they seem to be adaptable to a distributed setting. 

Johnson and Kargupta [4] develop a distributed hierarchical clustering algorithm on 
heterogeneously distributed data. It first generates local cluster models and then com- 
bines these into a global model. At each local site, the chosen hierarchical clustering 
algorithm is applied to generate local dendograms which are then transmitted to a cen- 
tral site. Using statistical bounds, a global dendogram is generated. 

Lazarevic et al. [21] consider the problem of combining spatial clusterings to pro- 
duce a global regression-based classifier. They assume homogeneously distributed data 
and that the clustering produced at each site has the same number of clusters. Each lo- 
cal site computes the convex hull of each cluster and transmits the hulls to a central site 
along with regression model for each cluster. The central site averages the regression 
models in overlapping regions of the hulls. 

Samatova et al. [26] develop a method for merging hierarchical clusterings from 
homogeneously distributed, real-valued data. Each site produces a dendogram based on 
local data, then transmits it to a central site. To reduce communication costs, they do not 
send a complete description of each cluster in a dendogram. Instead an approximation 
of each cluster is sent consisting of various descriptive statistics e.g. number of points 
in the cluster, average square Euclidean distance from each point in the cluster to the 
centroid. The central site combines the dendogram descriptions into a global dendogram 
description. 

Strehl and Ghosh [29] develop methods for combining cluster ensembles in a cen- 
tralized setting. They argue that the best overall clustering maximizes the average nor- 
malized mutual information over all clusters in the ensemble. However, they report that 
finding a good approximation directly is very time-consuming. Instead they develop 
three more efficient algorithms which are not theoretically shown to maximize mutual 
information, but are empirically shown to do a decent job. Given n data points and N 
clusterings (clustering i has ki clusters), consider an n x (Y^iL i &i) matrix H con- 
structed by concatenating the collection of n x ki matrices Hi for each clustering. The 
(£, j) entry of Hi is one if data point l appears in cluster j in clustering i, otherwise zero. 
One algorithm simply applies any standard similarity based clustering over the follow- 
ing similarity matrix — ^ . The (p. q) entry is the fraction of clusterings in which data 
point p and q appear in the same cluster. The other two algorithms apply hyper-graph 
based techniques where each column of H is regarded as a hyperedge. 
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In principle, Strehl and Ghosh’s ideas can be readily adapted to heterogeneously 
distributed data (they did not explicitly address this issue). Each site builds a local clus- 
tering, then a centralized representation of the H matrix is constructed. To compute H 
directly, each site sends Hi to a central site. This, however, likely will involve too much 
communication on datasets with large numbers of tuples (n) because Hi is n x For 
Strehl and Ghosh’s ideas to be adapted to a distributed setting, the problem of construct- 
ing an accurate centralized representation of H using few messages need be addressed. 

Fred and Jain [7] report a method for combining clusterings in a centralized set- 
ting. Given N clusterings of n data points, their method first constructs an n x n, 

TT ZjT 

co-association matrix (the same as ™ as described in [29]). Next a merge algorithm 
is applied to the matrix using a single link, threshold, hierarchical clustering technique. 
For each pair (i,j) whose co-association entry is greater than a predefined threshold, 
merge the clusters containing these points. 

In principal Fred and Jain’s approach can be adapted to heterogeneously distributed 
data (they did not address the issue). Each site builds a local clustering, then a central- 
ized co-association matrix is built from all clusterings Like Strehl and Ghosh’s ideas; 
in order for Fred and Jain’s approach to be adapted to a distributed setting, the problem 
of building an accurate co-association matrix in a message efficient manner must be 
addressed. 

Jouve and Nicoloyannis [14] also develop a technique for combining clusterings. 
They use a related but different approach than those described earlier. They reduce the 
problem of combining clusterings to that of clustering a centralized categorical data 
matrix built from the clusterings and apply a categorical clustering algorithm (KER- 
OUAC) of their own. The categorical data matrix has dimensions n x N and is defined 
as follows. Assume clustering 1 < i < N has clusters labeled 1, 2, . . . , fcj. The (j, i ) 
entry is the label of the cluster (in the i th clustering) containing data point j. The KER- 
OUAC algorithm does not require the user to specify the number of clusters desired in 
the final clustering. Hence, Jouve and Nicoloyannis’ method does not require the de- 
sired number of clusters in the combined clustering to be specified. 

Like the approaches in [29] and [7], Jouve and Nicoloyannis’ technique can be read- 
ily adapted to heterogeneously distributed data. A centralized categorical data matrix is 
built from the local clusterings, then the central site applies KEROUAC (or any other 
categorical data clustering algorithm). However, the problem of building an accurate 
matrix in a message efficient manner must be addressed (despite the fact that their title 
contains “Applications for Distributed Clustering”, they did not address the issue). 

Topchy et al. [30] develop an intriguing approach based on combining many weak 
clusterings in a centralized setting. One of the weak clusterings used projects the data 
onto a random, low-dimensional space (1 -dimensional in their experiments) and per- 
forms I\ -means on the projected data. Then, several methods for combining clusterings 
are used based on finding a new clustering with minimum sum "difference” between 
each of the weak clusterings (including methods from [29]). His idea does not seem di- 
rectly applicable to a distributed setting where reducing message communication is the 
central goal. Hence, the work saved at each site by producing a weak clustering is not of 
much importance. However, he discusses several new ideas for combining clusterings 
which are of independent interest. For example, he shows that when using generalized 
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mutual information, maximizing the average normalized mutual information consensus 
measure of Strehl and Ghosh is equivalent to minimizing a square-error criterion. 

Merugu and Ghosh [22] develop a method for combining generative models pro- 
duced from homogeneously distributed data (a generative model is a weighted sum of 
multi-dimensional probability density functions i.e. components). Each site produces a 
generative model from its own local data. Their goal is for a central site to find a global 
model from a pre-defined family (e.g. multivariate, 10 component Gaussian mixtures), 
which minimizes the average Kullback-Leibler distance over all local models. They 
prove this to be equivalent to finding a model from the family which minimizes the KL 
distance from the mean model over all local models (point-wise average of all local 
models). 

They assume that this mean model is computed at some central site. Finally the 
central site computes an approximation to the optimal model using an EM-style algo- 
rithm along with Markov-chain Monte-carlo sampling. They did not discuss how the 
centralized mean model was computed. But, since the local models are likely to be con- 
siderably smaller than the actual data, transmitting the models to a central site seems to 
be a reasonable approach. 

Januzaj et al. [13] extend a density-based centralized clustering algorithm, 
DBSCAN, by one of the authors to a homogeneously distributed setting. Each site car- 
ries out the DBSCAN algorithm, a compact representation of each local clustering is 
transmitted to a central site, a global clustering representation is produced from local 
representations, and finally this global representation is sent back to each site. A clus- 
tering is represented by first choosing a sample of data points from each cluster. The 
points are chosen such that: (i) each point has enough neighbors in its neighborhood 
(determined by fixed thresholds) and (ii) no two points lie in the same neighborhood. 
Then K -means clustering is applied to all points in the cluster, using each of the sample 
points as an initial centroid. The final centroids along with the distance to the furthest 
point in their A'-means cluster form the representation (a collection point, radius pairs). 
The DBSCAN algorithm is applied at the central site on the union of the local repre- 
sentative points to form the global clustering. This algorithm requires an e parameter 
defining a neighborhood. The authors set this parameter to the maximum of all the 
representation radii. 

Methods [13], [22], and [26] are representatives of the centralized ensemble-based 
methods. These algorithms focus on transmitting compact representations of a local 
clustering to a central site which combines to form a global clustering representation. 
The key to this class of methods is in the local model (clustering) representation. A 
good one faithfully captures the local clusterings, requires few messages to transmit, 
and is easy to combine. 

Both the ensemble approach and the multiple communication round-based cluster- 
ing algorithms usually work a lot better than their centralized counterparts in a 
distributed environment. This is well documented in the literature. While, the DDM 
technology requires further advancement for dealing with peer-to-peer style and het- 
erogeneous data, the current collection of algorithms offer a decent set of choices. The 
following section organizes the distributed clustering algorithms based on the data dis- 
tribution (homogeneous vs. heterogeneous) they can handle. 




Multi-agent Systems and Distributed Data Mining 



11 



Homogeneous vs. Heterogeneous Clustering Literature. A common classification 
of DDM algorithms in the literature is: those which apply to homogeneously distributed 
(horizontally partitioned) or heterogeneously distributed (vertically partitioned) data. 
To help the reader sort out the clustering methods we have described, we present the 
four-way classification seen in Table 4.2. 
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Ensemble 
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Fig. 5. Four-way clustering algorithms classification 



The following section considers a specific instance of a DDM problem — analyzing 
data in a sensor network with peer-to-peer communication architecture. It identifies 
some of the constraints in clustering data in such environments, offers a perspective of 
the existing distributed clustering algorithms in the context of this particular application, 
and points out areas that require further research. 



5 Sensor Networks, Distributed Clustering, and Multi-agent 
Systems 

Sensor networks are finding increasing number of applications in many domains, 
including battle fields, smart buildings, and even human body. Most sensor networks 
consist of a collection of light-weight (possibly mobile) sensors connected via wire- 
less links to each other or to a more powerful gateway node that is in turn connected 
with an external network through either wired or wireless connections. Sensor nodes 
usually communicate in a peer-to-peer architecture over an asynchronous network. In 
many applications, sensors are deployed in hostile and difficult to access locations with 
constraints on weight, power supply, and cost. Moreover, sensors must process a contin- 
uous (possibly fast) stream of data. The resource-constrained distributed environments 
of the sensor networks and the need for collaborative approach to solve many of the 
problems in this domain make multi-agent systems-architecture an ideal candidate for 
application development. For example, a multi-agent sensor-network application utiliz- 
ing learning algorithms is reported in [27], This work reports development of embedded 
sensors agents used to create an integrated and semi-autonomous building control sys- 
tem. Agents embedded on sensors such as temperature and light-level detectors, move- 
ment or occupancy sensors are used in conjunction with learning techniques to offer 
smart building functionalities. The peer-to-peer communication-based problem solving 
capabilities are important for sensor networks and there exists a number of multi-agent 
system-based different applications that explored these issues. Such systems include: 
an agent based referral system for peer-to-peer(P2P) file sharing networks [32], and 
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an agent based auction system over a P2P network [23]. A framework for developing 
agent based P2P systems is described in [1], Additional work in this area can be found 
elsewhere [27,28,6]. The power of multi-agent-systems can be further enhanced by 
integrating efficient data mining capabilities and DDM algorithms may offer a better 
choice for multi-agent systems since they are designed to deal with distributed systems. 

Clustering algorithms are likely to play an important role in many sensor-network- 
based applications. Segmentation of data observed by the sensor nodes for situation 
awareness, detection of outliers for event detection are only a few examples that may 
require clustering algorithms. The distributed and resource-constrained nature of the 
sensor-networks demands a fundamentally distributed algorithmic solution to the clus- 
tering problem. Therefore, distributed clustering algorithms may come handy [18] when 
it comes to analyzing sensor network data or data streams. 

Clustering in sensor-networks offers many challenges, including, 

1 . limited communication bandwidth, 

2. constraints on computing resources, 

3. limited power supply, 

4. need for fault-tolerance, and 

5. asynchronous nature of the network 

Distributed clustering algorithms for this domain must address these challenges. The 
algorithms discussed in the previous section addresses some of the issues listed above. 
For example, most of these distributed clustering algorithms are lot more communi- 
cation efficient compared to their centralized counterparts. There exists several exact 
distributed clustering algorithms, particularly for homogeneous data. In other words, 
the outcome of the distributed clustering algorithms are provably same as that of the 
corresponding centralized algorithms. For heterogeneous data, the number of choices 
for distributed clustering algorithms is relatively limited. However, there do exist sev- 
eral techniques for this latter scenario. Most of the distributed clustering algorithms are 
still in the domain of academic research with a few exceptions. Therefore, the scala- 
bility properties of these algorithms are mostly studied for moderately large number 
of nodes. Although the communication-efficient aspects of these distributed clustering 
algorithms help addressing the concerns regarding restricted bandwidth and power sup- 
ply, the need for fault-tolerance and P2P communication-based algorithmic approach 
are yet to be adequately addressed in the literature. 

The multiple communication round-based clustering algorithms described in 
Section 4 involve several rounds of message passing between nodes. Each round can 
be thought of as a node synchronization point (multiple sensor synchronizations are 
required). This may not go very well in a sensor network-style environment. 

Centralized ensemble-based algorithms provide us with another option. They do not 
require global synchronization nor message passing between nodes. Instead, all nodes 
communicate a model to a central node! which combines the models). In absence of 
a central controlling site one may treat a peer as a central combiner and then apply 
the algorithms. We can envision a scenario in which an agent at a sensor node initi- 
ates the clustering process and as it is the requesting node, it performs the process of 
combining the local cluster models received from the other agents. However, most of 
the centralized ensemble-based method algorithms are not specifically designed to deal 
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with stream data. That is something that we may need to address in the immediate fu- 
ture. Algorithms such as [13], [22], [26] deal with the limited communication issue by 
transmitting compact, lossy models (rather than complete specifications of the cluster- 
ings), which may be necessary for a sensor-network-based application. The following 
section concludes this paper. 



6 Conclusions 

Multi-agent systems are fundamentally designed for collaborative problem solving in 
distributed environments. Many of these application environments deal with empirical 
analysis and mining of data. This paper suggests that traditional centralized data mining 
techniques may not work well in many distributed environments where data centraliza- 
tion may be difficult because of limited bandwidth, privacy issues and/or the demand 
on response time. 

This paper pointed out that distributed data mining algorithms may offer a better 
solution since they are designed to work in a distributed environment by paying careful 
attention to the computing and communication resources. The paper focused on dis- 
tributed clustering algorithms and their applications in sensor networks just to illustrate 
some of the existing challenges and weaknesses of the DDM algorithms. It noted that 
while these algorithms usually perform way better than their centralized counter-parts 
on grounds of communication efficiency and power consumption, there exist several 
open issues. Developing peer-to-peer versions of these algorithms for asynchronous net- 
works and paying attention to fault-tolerance are some examples. Nevertheless, existing 
pleasures of distributed clustering algorithms do provide a reasonable class of interest- 
ing choices for the next generation of multi-agent systems that may require analysis of 
distributed data. 
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Abstract. To realize large-scale socially embedded multiagent systems, this 
paper proposes a new system design methodology towards society-centered 
design. We have already developed the scenario description language Q, 
which describes interaction protocols that link agents to society. Using the 
virtual space called FreeWalk, wherein agents behave under given Q scenarios, 
we explain each step of society-centered design. The process consists of 
participatory simulation, where agents and human-controlled avatars coexist in 
virtual space to jointly perform simulations, and augmented experiment, where 
an experiment is performed in real space by human subjects, scenario- 
controlled agents, and human extras. As an application of society-centered 
design, we are working on mega navigation, where millions of humans and 
socially embedded agents collaborate for developing services for traffic control, 
crisis management and large-scale event navigation. 



1 Introduction 

The waterfall model has been used as a software development model for a long time. 
Given the increase in human-computer interaction, however, it has become essential 
to use the user-centered design approach when creating usable and accessible 
interactive systems. It is natural to ask whether or not we need a different model for 
mobile/ubiquitous/pervasive computing, where thousands or millions of computers or 
electronic devices are connected in an ad hoc manner. Agent-based software 
engineering has been intensively studied [12]. Large scale multiagent systems may be 
a natural solution [14,20]. The problem is how to ensure that they yield adequate 
behavior: “adequate behavior” does not merely mean computationally correct 
behavior, but appropriate social behavior when embedded in human societies. 

It is known that the behavior of socially embedded systems is hard to predict, not 
only because the system is highly distributed, but also because the system is exposed 
to the impact of human interaction. In this paper, in contrast to user-centered design, 
we pursue society-centered design, where participatory technologies are applied to 
confirm the adequateness of socially embedded systems; the following steps provide a 
description of this approach. 

1 . Describe interaction protocols linking service agents and users so as to define the 
expected behaviors of socially embedded agents. We have been developing the 
scenario description language Q [10] for this purpose. 
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2. Perform multiagent simulation by modeling the autonomous behaviors of service 
agents and users under the given protocols. The simulation takes place in virtual 
space by service agents and simulated users (we call both agents). Results of the 
simulation can estimate how the entire system would work in society. 

3. Replace some of the simulated users by human-controlled avatars to perform a 
participatory simulation ; the human subjects directly control their avatars. The 
simulation is performed in virtual space, and the avatars are controlled by the 
humans sitting in front of their desktop computers. From the results of the 
participatory simulation, we can improve the models of service agents and users, 
as well as the interaction protocols. 

4. Perform experiments in a real space to try out the entire system with human subjects. 
Since the number of human subjects is often limited, however, the experiment 
should be augmented by scenario-controlled agents and human extras. We called 
this the augmented experiment ; note that most of participants are virtual constructs. 

Drogoul el al. proposed a methodological process for developing multiagent-based 
simulations, and introduced the idea of the participatory design of simulations [5], 
Bousquet el al. applied role games to modeling multiagent systems [3], In this paper, 
we propose participatory technologies to conduct simulations and experiments for 
large-scale socially embedded systems. 

In the following sections, we explain the society-centered design approach in detail, 
through visualizing it by using FreeWalk [ 16], a virtual space platform developed by the 
digital city project in Kyoto [11]. At the end of this paper, we apply the above steps to 
design mega navigation ; the goal is to create services for traffic control, crisis 
management and large-scale event navigation where millions of people may be involved. 

2 Scenario Engineering 

In this section, we focus on interaction protocols to link agents, not on the internal 
mechanisms of the agents. We view interaction protocols as behavioral guidelines of 
socially embedded agents, while agents keep certain degree of autonomy under the 
given constraints. We call the description of interaction protocols as scenarios. For 
realistic applications, since scenario writers are often not computing professionals, the 
correctness of scenarios for multiple agents is not guaranteed. Problems in scenarios 
should be adjusted not by proving their correctness but by conducting rehearsals. To 
monitor rehearsals, visualizing the scenario execution processes becomes essential. 

Q is a scenario description language for multiagent systems that allows us to define 
how agents are expected to interact with each other. Q’s language functionality is 
summarized as follows. 

Cues and Actions 

An event that triggers interaction is called a cue. Cues are used to request agents to 

observe their environment. No cue is permitted to have any side effect. Cues keep 

on waiting for the event specified until the observation is completed successfully. 

Compared to cues, actions are used to request agents to change their environment. 
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(defscenario card14 () 

(opning 

(#t 

(Ispeak "Hm-hum, you are so enthusiastic.") 
(Ispeak "Then, how about this page?") 

(! display :url "http://kimono.com/index.htm") 

(go reactions-to-user))) 

(reactions-to-user 

((?watch_web :url "http://kimono.com/type.htm") 
(Ispeak "There are many types of obi.") 

(Ispeak "Can you tell the difference?") 

(go reactions-to-user)) 

((?watch_web :url "http://kimono.com/fukuro.htm") 
(Igesture animation "GestureLeft") 

(Ispeak "Fukuro obi is for a ceremonial dress.") 
(Ispeak "Use it at a dress-up party!") 

(go reactions-to-user)) 

((?watch_web :url "http://kimono.com/maru.htm") 
(card42 self) 

(go reactions-to-user)) 

((?timeout dime 20) 

(go closing))) 

(closing 

(#t 

(Ispeak "Did you enjoy Japanese Kimono?") 
(Ispeak "OK, let’s go to the next subject.")))) 




Designed by Tomasan Kimono School 



a) A sample scenario in Q language 



Card ID 


14 


Card Name |Visiting Kimono Web site | Card Type |User Initiative 




Action 


Opening 


Hm-hum, you are so enthusiastic. 
Then, how about this page? 
http://www.kimono.com/index.htm 






Cue 


Action 


Reactions to 
Users’ 

Mouse Click 
( Repeat ) 


Mouse 

Click 


http://kimono.com/type.htm 


There are many types of obi. 
Can you tell the difference? 


http://kimono.com/fukuro.htm 


(GestureLeft) 

Fukuro obi is for a ceremonial dress. 
Use it at a dress-up party! 




http://kimono.com/maru. htm 


(Evaluate Card42) 




No 


Seconds 


Action 




Reaction 


20 


(End of Repeat) 




Action 


Closing 


Did you enjoy Japanese Kimono? 

OK, let’s move on to the next subject. 



b) The same scenario in IPC representation 



Fig. 1 . Q Scenario and Interaction Pattern Card (with Y. Murakami and A. Yamamoto) 

Scenarios 

Guarded commands are introduced for the situation wherein we need to observe 
multiple cues simultaneously. A guarded command combines cues and actions; 
after one of the cues becomes true, its corresponding actions are performed. A 
scenario is used for describing protocols in the form of an extended finite state 
machine, where each state is defined as a guarded command. Scenarios can be 
called from other scenarios. 
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Agents and Avatars 

Agents, avatars and a group of agents can be defined. An agent is defined by a 
scenario that specifies what the agent is to do. Even if a group of agents executes 
the same scenario, the agents exhibit different actions as they interact with their 
local environment (including other agents and avatars). Avatars are controlled by 
humans, and do not usually require any scenario. 

Figure 1(a) shows an example scenario for Microsoft agents. Agents can be 
autonomous or dependent. If autonomous, scenarios cannot be complex; if not, 
scenario writers can specify all details. Note that the granularity of cues and actions 
depends on two independent factors: the level of agent autonomy, and the degree of 
preciseness required by the scenario writer. 

Figure 2 shows the Q architecture for handling scenarios. When a scenario is given 
to a particular agent, a Q processor is instantiated and coupled to the corresponding 
agent system. The agent system is usually capable of hosting multiple agents. For 
example, the three dimensional virtual space FreeWalk can support hundreds of 
agents simultaneously. Consequently, the Q interpreter is designed to execute 
hundreds of scenarios simultaneously. Though the Q processor is implemented at the 
top of the Scheme, the program interface supports C++ and JAVA software, so it is 
easy to add Q to legacy agent systems. Q has been already combined with Microsoft, 
FreeWalk, CORMAS [2,22] and Caribbean [23,24] agents. 
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Fig. 2. Q Architecture 



In order to develop successful socially embedded systems, scenario writers must 
provide appropriate scenarios to agents. It is necessary to establish a procedure that 
models agents at an appropriate level of abstraction. We propose the following three- 
step procedure for creating scenarios [15]. 
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STEP1: Define a Vocabulary 

A scenario writer and an agent system developer agree upon cues and actions as 
the interface between them. Note that cues and actions are not provided a priori by 
Q language but are defined for each application domain. 

STEP2: Write Scenarios 

The scenario writer describes scenarios using Q language, while the agent system 
developer implements cues and actions. 

STEP3: Extract Interaction Patterns 

Patterns of interaction are extracted from already written scenarios, and are used to 
define Interaction Pattern Cards (IPC). Scenario writers then describe scenarios 
by using the cards, and use their experience to further improve the definition of the 
cards. 

Figure 1(b) indicates an IPC representation that is the exactly equivalent of the Q 
scenarios in Figure 1(a). Note that IPC is not merely an Excel representation of Q 
language; it is a language to express interaction patterns in a particular domain. The 
introduction of IPC not only provides a simple means of writing scenarios, but also 
facilitates dialogs between scenario writers and agent system developers. 

3 Participatory Simulation 

There are two types of multiagent-based simulations depending on their purposes; a) 
analytic multiagent-based simulations with a simple internal model of agents 
(hereafter referred to as analytic simulations) and b) synthetic multiagent-based 
simulation with a complex internal model of agents (hereafter referred as synthetic 
simulation). Analytic simulations have been used to analyze complex social systems. 
Here, the KISS principle (Keep It Simple, Stupid) is often applied [1], The KISS 
principle states that agent modeling should be simple even though the observed 
phenomenon is complex, and that complexity should be a result of agent interaction. 
Hence, agents are expressed using a simple computational model that incorporates 
limited parameters. This approach is mainly used in the analysis of the relationship 
between the macro properties of the entire system and the micro properties of the 
agents constituting the system [7], On the other hand, synthetic simulation is used for 
the reproduction of reality-based situations. Agent models reflecting the real world 
are created to make the simulation as realistic as possible. This approach is used in an 
early stage of system development [13], in the examination of strategies for decision 
making, and in education or training [19]. 

In our society-centered design approach, we first conduct synthetic multiagent- 
based simulations, and then replace some of the agents by human-controlled avatars: 
humans directly control agents in multiagent-based simulations. We call simulation 
including human-controlled avatars as multiagent-based participatory simulation 
(hereafter referred to as participatory simulation ). 

Figure 3 illustrates how to realize participatory simulations. Since the scenario 
processor interprets interaction protocols and requests agents to perform sensing and 
acting functions, it is easy to realize participatory simulation by replacing scenario- 
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controlled agents with human-controlled avatars. We can monitor the entire process of 
simulation by visualizing the virtual space as described in Figure 3(b). Recording videos 
of behaviors of human subjects in real space is useful for analyzing simulation results. 



A M Human 

^ 'fl' Subject 

Real Space 



ited 



b) Multiagent-Based Participatory Simulation 
Fig. 3. Participatory Simulation 

We conducted a participatory simulation in the evacuation domain. We simulated 
the controlled experiments conducted by Sugiman in 1988 [21]. He established a 
simple environment with human subjects to determine the effectiveness of two 
evacuation methods: the Follow-direction method and the Follow-me method. In the 
former, the leader shouts out evacuation instructions and eventually moves toward the 
exit. In the latter, the leader tells a few of the nearest evacuees to follow him and 
actually proceeds to the exit without verbalizing the direction of the exit. Sugiman 
used university students as evacuees and monitored the progress of the evacuations 
with different number of leaders. The experiment was held in a basement that was 
roughly ten meters wide and nine meters long; there were two exits, one of which was 
not obvious to the evacuees. The ground plan of the basement is shown in Figure 4. 

Our first evacuation simulation used scenarios with simulated leaders and evacuees in 
a two-dimensional virtual space as displayed in Figure 4(a). This simulation showed that 
the Follow-me method outperforms the Follow-direction method when there are enough 
leaders. Since the simulation results closely parallel those recorded in Sugiman’ s 
controlled experiment, we could conclude that the agents were reasonably well modeled 
[15]. We then examined the same scenario in the three-dimensional virtual space 
Free Walk. A difference between the two-dimensional and three-dimensional simulations 
is that the latter allows humans to join the simulation: human subjects can sense their 
environment and control their avatars in the virtual space. 

A participatory simulation was then performed with six human subjects: we replaced 
six out of twenty scenario-controlled agents by human-controlled avatars. The virtual 
space is designed so that human subjects cannot distinguish agents and avatars. By 
analyzing the logs and videos of simulations, we leaned that a) evacuees look for leaders 
not by their ears but by their eyes, b) when finding a leader, evacuees tend to approach 
the leader, even if the Follow-direction method is performed, and c) when losing their 
leaders, evacuees follow other evacuees moving in the same direction. 




Scenario 

pnPn 

Virtual Space » 

a) Multiagent-Based Simulation 
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Once an accurate model is acquired by participatory simulations, it becomes 
possible to simulate an experiment that has not conducted in real space. For example, 
Sugiman’s experiment was performed by two or four leaders, but we can vary the 
number to clarify the relation between the number of leaders and the time required for 
evacuation. Moreover, we can explore an effective combination of the Follow-me and 
Follow-direction methods. 

Participatory simulations are useful to educate or train people once we obtain 
accurate agent models and interaction protocols. It provides people with a vicarious 
experiential learning environment wherein evacuation or other drills can be 
experienced. 




b) 3D Simulation 

Fig. 4. Evacuation Simulation (with T. Kawasoe and K. Minami) 




Many technical issues remain with participatory simulations. First, time management 
becomes difficult in participatory simulations. In our current design of multiagent-based 
simulations, the scenario interpreter has no responsibility for time management: time is 
controlled by the agent systems executing the sensing and acting functions. Therefore, in 
FreeWalk/Q, just as any combination between Q and other agent systems, only 
FreeWalk manages time. To make this reasonable, however, we should assume that the 
scenario processor runs far more rapidly than the agent systems executing sensing and 
acting functions. This assumption becomes problematic if we use high-performance grid 
platforms as agent systems [8]. Furthermore, since human subjects play the simulation 
in real space, we need to keep time consistency among virtual and real spaces. 

Another issue is that learning from human subjects is not straightforward. By 
analyzing the logs of participatory simulation, we can improve the interaction 
protocols and agent models in several different ways. For example, in the evacuation 
domain, if the participatory simulation results differ from the expectation, there are 
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several choices: to modify a given interaction protocol, such as Follow-me or Follow- 
direction methods, to modify simulated human models, or to regard the results just as 
an exception. To appropriately learn lessons from simulation results, we should 
carefully analyze the data. Machine learning can help this analysis, and so will 
become a central issue in participatory simulation. 



4 Augmented Experiment 

To understand how people accept/reject socially embedded systems, many real-world 
experiments have been conducted. A well-known example involves video phones. 
Since the value of video phones depends on the number of users, and user behavior in 
everyday life is not easy to simulate in virtual space, it is essential to observe how 
users accept and utilize the new technology. In mobile/ubiquitous/pervasive 
computing, however, because of a large number of electronic devices are embedded 
in human society, it is costly or often impossible to conduct experiments in real space. 
Augmented experiments have the concept of performing experiments with a small 
number of human subjects in real space with augmentation by multiagent systems. 
We call such experiments multiagent-base augmented experiments (hereafter referred 
to as augmented experiments). 



• • 

Real Space 



Human 

Subject 



t 



a) Real-World Experiment 




Real Space 

b) Multiagent-Based Augmented Experiment 



Fig. 5. Augmented Experiment 



Figure 5 illustrates how to realize augmented experiments. To provide enough 
reality to the human subjects, we need human extras around the subjects. In contrast 
to participatory simulations, the human extras do not control avatars, while agents in 
virtual space control the human extras. To monitor an entire experiment, the behavior 
of human subjects should be reproduced in virtual space. This can be realized if the 
real space is equipped with enough sensors. To analyze the entire experiment, 
however, video capture of the real space is indispensable in augmented experiments. 

The augmented experiment approach is not effective for controlled experiments 
with just a few subjects; instead, human subjects would be used rather than extras and 
complex virtual systems. For testing evacuation methods in open space like railway 
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stations, however, we need augmented experiments, since there is no other way to 
conduct realistic experiments. To perform augmented experiments, we have placed 
more than twenty cameras in Kyoto subway stations, and successfully captured the 
movements of passengers in real time. We then reproduced their behavior in virtual 
space. We have implemented a monitoring system based on transcendent 
communication [17]. The bird’s-eye view of real space is reproduced on the screen of 
the control center so as to monitor the experiment. Figure 6 is a snapshot of a 
monitoring system; evacuees on a subway station platform are projected in virtual 
space. A leader in the control center can easily monitor the experiment. Furthermore 
the leader can point at particular passengers on the screen, and talk to them through 
mobile phones. When the monitor detects pointing operations, the audio connection is 
immediately activated between the control center and indicated passengers. A 
dragging operation indicating a rectangular area enables the leader to broadcast an 
announcement to a group of passengers. 




Fig. 6. Transcendent Communication at Kyoto Subway Station (with H. Nakanishi and H. Ito) 

The difficulty with augmented experiments is providing the human subjects with a 
sufficient level of reality. This is not easy if the number of human extras is limited. If 
there are not enough extras, their roles should be reassigned during the experiment. To 
control human extras by scenarios in a timely fashion, agents need to receive sensing 
information from the corresponding human extras. This is possible by embedding 
sensors into the experiment environment. Another difficulty is how to evaluate the 
results of experiments. The evaluation of agent models and interaction scenarios can be 
done by comparing the results of augmented experiments to those of multiagent-based 
simulations. Learning issues are similar to participatory simulations, but the evaluation 
is more difficult, since the behaviors of human subjects in real space are hard to analyze. 

5 Example: Mega Navigation 

To apply society-centered design to large scale multiagent systems, we started a project 
called mega navigation ; it assumes that at least a million agents are embedded in society. 
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One agent serves one human by guiding him/her for traffic control, crisis management 
and large-scale event navigation. To draw a clear image of mega navigation, examples 
of the Beijing Olympics in 2008 are listed below. We assume that third generation 
mobile phones with GPS will be in wide use in Beijing at that time. 

Restaurant Recommendation: Suppose one company is planning to develop a 
restaurant recommendation service for foreign tourists. Tourists can get his 
location by GPS and can take a photo of the restaurants visited and their 
recommendations. The company expects tourists to provide the recommendations 
voluntarily. The question is whether or not the tourists will participate and 
upload/download recommendations. Since the information available can be 
consumed at no cost (the abuse of the commons), the response seems quite 
personal. 

Smart Tag: This company is also developing a smart tag for tourists so that they can 
easily find new friends who have similar hobbies, preferences, and schedules. Can 
we expect tourists to store their personal information in the tag? The company 
believes that the smart tag will help tourists to create their own communities, which 
would make the restaurant recommendation service community-oriented. 

Traffic Control: Traffic jams will increase in Beijing during the Olympics. But 
nobody knows how crowded the roads and Olympic facilities will become. The 
company is planning to guide people to less crowded roads and venues by using 
data of the tourists’ smart tags. Will tourists follow the instructions? To decrease 
traffic jams in Beijing, demand responsive bus services might be a good idea. 
However, it is hard to know how tourists will respond to this service. 

Crisis Management: Suppose a disaster happens during the Olympics. The company 
provides navigation service for people who have GPS mobile phones. It is expected 
that people with mobile phones send their locations and telephone numbers to the 
company. At the control center, the locations of such people are displayed on a 
large screen. Using transcendent communication, the control center talks to people 
to get local information, and asks them to guide surrounding people to safe areas. 
Will people voluntarily join this service? How many GPS mobile phone users are 
necessary to provide safe guidance to all citizens in Beijing? 

Research on large-scale multiagent systems has just started [18]. To handle a million 
people, we have been developing a mega navigation platform by combining Q and 
Caribbean [21,22]. Caribbean is a large-scale agent server designed for handling a 
million agents. Agents are created in the Caribbean server and persist in the server. 
Caribbean agents communicate with other agents via asynchronous peer-to-peer 
messaging. When a message is delivered to a particular agent, the corresponding activity 
of the agent is invoked. Agents can invoke service objects, which provide common 
services such as database management. Figure 7 shows how we connect Q and 
Caribbean. As in the case of FreeWalk/g, time management is performed by Caribbean. 
Since Caribbean is a mega-scale agent server, however, it appears that Q cannot 
guarantee to run scenarios faster than Caribbean. To make time management easier, we 
are currently developing a translator to turn Q scenarios into Caribbean activities. 
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We plan to use Caribbean/<2 to conduct participatory simulations and augmented 
experiments on the service examples listed above. Traffic control problems [6] and 
demand bus services [4] are common examples in the agent research community. 
Multiagent-based simulation for demand responsive bus services is straightforward: 
we first define an interaction protocol between passengers and bus companies, run 
simulation with agents that simulate passengers, bus drivers and bus companies. 




Fig. 7. Mega Navigation Platform 

The corresponding participatory simulations are not straightforward. We want to 
focus on passengers’ decision making. It is not obvious, though, how human subjects 
in front of desktop computers can make realistic decisions given several options: wait 
for a bus, give up and walk, take a taxi, etc. Augmented experiments, on the other 
hand, would be enjoyable. A human subject with a mobile phone would actually 
make a phone call to the bus control center. One bus is allocated for him/her in virtual 
space, but the human subject cannot ride a virtual bus. To provide enough reality to 
the human subject, a real vehicle (maybe a chartered taxi if the budget is limited) is 
used to realize the virtual bus. The human subject then rides on the bus. Several 
simulated users may share the bus with the human subject. The bus stops at several 
places to let the simulated users get off the bus. Finally, the bus stops at the 
destination of the human subject, lets him/her get off, and the bus becomes virtualized 
afterwards. The real vehicle then becomes available to realize another virtual bus. 



6 Conclusions 

In this paper, we proposed society-centered design for making large-scale agent systems 
run in human societies. We proposed a paradigm shift from user-centered design, where 
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individual usability is a major concern, to society-centered design, where usability in the 
community becomes essential. Using multiagent-based simulations, we invented a system 
design process including participatory simulations and augmented experiments so as to 
confirm the adequateness of socially embedded multiagent systems. FreeWalk/G is used to 
visualize those steps of society-centered design. Though the concept of society-centered 
design is open-ended, we proposed to include the following three components. 

Scenario Engineering: We developed Q, a scenario description language, to define 
interaction protocols that link agents and humans. Scenarios also foster the 
emergence of dialogs between agent designers (computing professionals) and 
application designers (scenario writers). Q has been combined with several legacy 
agent systems: Microsoft, FreeWalk, CORMAS and Caribbean agents. The reason 
that totally different agent systems can be supported is because Q is designed to 
describe interactions among legacy agents. 

Participatory Simulation: We call multiagent-based simulations that include 
human-controlled avatars participatory simulations. In our framework, since agents 
are controlled by scenarios, it is easy to realize participatory simulations by 
replacing agents by human-controlled avatars. We can monitor the entire process of 
a participatory simulation by visualizing the virtual space. 

Augmented Experiment: We call real-world experiments augmented by simulated 
users augmented experiments. Given that mobile/ubiquitous/pervasive computing 
will see a large number of computers or electronic devices embedded in human 
society, it will be costly or impossible to conduct experiments in real space. 
Augmented experiments enable us to perform experiments with a small number of 
human subjects. 

We then introduced mega navigation as an example of society-centered design. We 
have started working on guide services for a million people who are connected via 
wireless networks. We implemented Caribbean/Q as a mega navigation platform. By 
using examples of services for the Beijing Olympics, we showed the potential of 
applications such as traffic control, crisis management and large scale event navigation. 

Society-centered design focuses on the process to embed large-scale multiagent 
systems into human societies, and is suitable for designing community support 
systems [9]. This methodology differs from the Gaia methodology [25], which 
focuses on an early stage of designing multiagent systems mainly for enterprise 
applications. As more humans and agents coexist, the need for describing interaction 
scenarios becomes essential. If we allow agents to be completely autonomous, the 
whole system becomes hard to control. It is necessary to give social constraints as 
scenarios so that the behaviors of agents are reasonable. Remaining research issues 
include the topic of how agents learn to behave under given social constraints. 
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Although Semantic Web Services have recently received a great deal of attention, the 
use of semantics to describe distributed cooperating components is not new. Multi- 
Agent Systems have long addressed the problems of coordinating heterogeneous, au- 
tonomous components so that they can collaborate and achieve joint goals. Several 
MAS use semantic descriptions to achieve a shared understanding of communications 
(both the purpose of the communication, and the contents of the communication); how- 
ever these solutions typically enable only a modest number of disparate MAS to collab- 
orate. With current advances in the Semantic Web, and Semantic Web Services, it may 
be possible to relax these constraints. 

The DARPA Agent Markup Language for Services (DAML-S) was proposed as 
a solution to discovering and automating the composition of services. Two years on 
(and a name change to OWL-S), it has gained widespread interest; yet is viewed as a 
Web Services proposal augmented through semantics, rather than a markup language 
for Agents. This paper presents an agent-oriented perspective of OWL-S, including an 
alternative set of usage metaphors. The emerging problems for agent-based adoption 
are discussed, as well as potential solutions that address this divergence. 
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Abstract. This paper presents an agent community based peer-to-peer 
information retrieval method called ACP2P method [16] and discusses 
the experimental results of the method. The ACP2P method uses agent 
communities to manage and look up information related to users. An 
agent works as a delegate of its user and searches for information that 
the user wants by communicating with other agents. The communication 
between agents is carried out in a peer-to-peer computing architecture. 
In order to retrieve information relevant to a user query, an agent uses 
a content file, which consists of retrieved documents and two histories : 
a query /retrieved document history (Q/RDH) and a query/sender agent 
history(Q/SAH). The former is a list of pairs of a query and the address 
of an agent that returned documents relevant to the query. The latter 
is a list of pairs of a query and the address of a sender agent and shows 
“who sent what query to the agent”. This is useful for finding a new 
information source. Making use of Q/SAH is expected to have a collabo- 
rative filtering effect, which gradually creates virtual agent communities, 
where agents with the same interests stay together. Our hypothesis is 
that a virtual agent community reduces communication loads necessary 
to perform a search. As an agent receives more queries, then more links 
to new knowledge are acquired. From this behavior, a “give and take” (or 
positive feedback) effect for agents seems to emerge. We implemented 
this method with Multi-Agent Kodama, and conducted experiments to 
test the hypothesis. The experimental results showed that the method 
employing two histories was much more efficient than a naive method em- 
ploying ’multicast’ techniques only to look up a target agent. Further, 
making use of Q/SAH facilitates bidirectional communications between 
agents and thus creates virtual agent communities. 



1 Introduction 

The rapid growth of the World Wide Web has made conventional search engines 
suffer from decreasing coverage in searching the Web. Internet users meet infor- 
mation floods every day, and are forced to filter out and choose the information 
they need. In order to deal with these problems, a lot of studies on distributed 
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information retrieval(e.g. [4,3]), information filtering (e.g. [14]), information rec- 
ommendation (e.g. [22]), expert finding(e.g. [28], [12]), or collaborative filtering 
(e.g. [9], [18], [10], [21]) have been carried out. Most systems developed in that re- 
search are, unfortunately, based on the server-client computational model and 
are often distressed by the fundamental bottle-neck coming from their central 
control system architecture. Although some systems based on the peer-to-peer 
(P2P for short) computing architecture (e.g. [24], [5], [8], [17]) have been developed 
and implemented, each node of most those systems only deals with simple and 
monolithic processing chores. 

Considering these issues, we proposed an Agent Community based P2P in- 
formation retrieval method (ACP2P method for short) [16]. The ACP2P method 
uses agent communities to manage and look up information related to a user 
query. An agent works as a delegate of its user and searches for information 
that the user wants by communicating with other agents. The communication 
between agents is carried out based on a P2P computing architecture. In order 
to retrieve information relevant to a user query, an agent uses two histories : a 
query/retrieved document history(Q/RDH for short) and a query/sender agent 
lristory(Q/SAH for short). The former is a list of pairs of a query and the address 
of the agent that returned documents relevant to the query, where the query was 
sent by the agent itself. The latter is a list of pairs of a query and a sender agent 
and shows “who sent what query to the agent” . This is useful for finding a new 
information source. Making use of the Q/SAH is expected to make a collabo- 
rative filtering effect emerge and to gradually create virtual agent communities, 
where agents with the same interests stay together. Our hypothesis is that a 
virtual agent community reduces communication loads necessary to perform a 
search. As an agent receives more queries, then more links to new knowledge 
are acquired. From this behavior, a “give and take” (or positive feedback) effect 
for agents seems to emerge. We implemented the method with Multi- Agent Ko- 
dama and conducted experiments to test the hypothesis, i.e., to evaluate how 
well Q/SAH works for reducing communication loads and for making a “give and 
take” effect emerge. This paper presents the ACP2P method and discusses the 
details of experimental results. The results showed that the method reduced com- 
munication loads much more than other methods which do not employ Q/SAH 
to look up a target agent, and was useful for making a “give and take” effect. 

The remainder of the paper is structured as follows. Section 2 considers the 
ACP2P method. Section 3 discusses the experimental results and Section 4 de- 
scribes related work. 



2 Agent Community Based Peer-to-Peer Information 
Retrieval Method 

2.1 Overview of ACP2P Method 

The ACP2P method employs three types of agents: user interface (UI) agent, 
information retrieval (IR) agent and history management (HM) agent. A set of 
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three agents (UI agent, IR agent, HM agent) is assigned to each user. Although a 
UI agent and an HM agent communicate only with the IR agent of their user, an 
IR agent communicates with other users’ IR agents not only in the community 
it belongs to, but also in other communities, to search for information relevant 
to its user’s query. A pair of Q/RDH and Q/SAH histories is managed by the 
HM agent. 




Fig. 1 . Agents and their Community Structure 

Fig. 1 shows an example of the agent community structure which the ACP2P 
method is based on. A portal agent in the figure is the agent which is a represen- 
tative of a community and manages all member agents’ addresses there, where 
a member agent of a community designates an IR agent. Unlike a super-peer of 
a super-peer network[27], the portal agent originally does not behave as a server 
of all member agents in the community, but mediates between a member agent 
and the others for advertising its joining the community or telling its messages 
to them. 

When a member agent wants to find any target agents which have information 
relevant to a query, the agent looks them up using a content file, which consists 
of retrieved documents, and two histories: Q/RDH and Q/SAH. The format of 
the content file and two histories will be described in the next section. 

If the target agents are found, a query is sent directly to them, and their 
retrieved results are also returned directly to the query sender IR agent. If the 
requested number of such agents is not found, the agent asks the portal agent 
to send the query to the all member agents in the community by a multicast 
technique. At that time, all the answers will be returned to the portal agent. 
If the number of results with a ’YES’ message reaches the requested number, 
without waiting for the rest of answers by other IR agents, the portal agent sends 
them back to the query sender IR agent. Even if the number of ’YES’ messages 
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did not reach the requested number after all IR agents replied, the portal agent 
also sends the currently held results to the query sender IR agent. 



4i 

user’ s query 
Ul Agent 



IR Agent 




HM Agent 



look up ‘query’ 



Answers 



History 


Q/RDH 

Q/SAH 


content 



target IR agents 
on the list 



7^ Portal Agent (PA) 



query! YES or No Answers 

©© © 

All IR agents in a community 



Fig. 2. Actions for Sending a Query 




IR Agent 



HM Agent 




Fig. 3. Actions for Receiving a Query 

Fig. 2, 3 and 4 show the processes or data flows in the following three cases 
: 1) an IR agent sends a query, 2) an IR agent receives a query from another 
IR agent or a portal agent, and 3) an IR agent receives answers from other IR 
agents, respectively. When receiving a query from a UI agent, an IR agent asks 
an HM agent to look up target agents with its history or a portal agent to do it 
using a query multicasting technique (Fig. 2). 

When receiving a query from other IR agents, the IR agent looks up the 
information relevant to a query, sends an answer to the query sender IR agent, 
and sends a pair of a query and the query sender IR agent’s address to an HM 
agent so that it can update Q/SAH (Fig. 3). 
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The returned answer to the query is either a pair of a ‘YES’ message and 
retrieved relevant documents or a ‘No’ message, which represents no relevant 
information, provided that retrieved documents are not returned when the query 
comes through a portal agent. 




Presents 
i Results 



©- © 

I Answers I 



Ul Agent 




IR Agent 



(query, Agent’ s Address, 

Answers) 



HM Agent 



I update ‘Q/RDH’ & Content I 



History 


Q/RDH 

Q/SAH 


content 



Fig. 4. Actions for Receiving Answers 

When receiving answers with a ‘YES’ message from other IR agents, an IR 
agent sends them to a UI agent, and sends them with a pair of a query and the 
addresses of answer sender IR agents to an HM agent to update Q/RDH (Fig. 4). 

2.2 Content File and History Files 

Table 1 shows the formats of a document content file: Content and two his- 
tories: Q/RDH and Q/SAH. The document content file consists of a list of 
4-tuples <title, text, original, range>, namely, the title of a document, its 
text content, the address of the IR agent whose user owns the document, and the 
allowed distribution range of the document, respectively. Documents retrieved 
and returned by other IR agents are shared into the Content file without any re- 
dundant registration. Thus the same content returned by two or more IR agents 

Table 1. The structures of Content file and two histories: Q/RDH and Q/SAH 



Content 


title 


the title of document 


text 


the content of document 


original 


the address of the IR agent whose user created the document 


range 


the range allowed to be distributed (ALL, Community, Agent) 


Q/RDH 


query 


queries sent by the agent recorded in the from field 


from 


the address of IR agent which has replied to the query in the 
query field 


Q/SAH 


query 


queries sent by the agent recorded in the from field 


from 


the address of the IR agent who sent the query in the query field 
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is registered only once into the Content file. Original documents, which are cre- 
ated by a user and initially assigned to his/her agent, also take the same format 
as the Content. 

The Q/RDH file comprises a list of pairs of < query, from>, each of which 
is a query sent by the agent itself and the address of IR agent that returned this 
retrieved information, respectively. The Q/SAH file is a list of pairs <query, 
from>, each of which is a query and the address of the agent which sent the 
query to the IR agent. Table 2 shows an example of part of a document content 
file. Table 3 also shows an example of part of Q/SAH file, which was originally 
written in Japanese. 



Table 2. A part of content file 



title 


text 


original 


range 


Netscape informal FAQ Japanese 
version 


HTML text in the file 


com_Netscape@ 


ALL 



Table 3. A part of Q/SAH 



query 


from 


telegram 

treatment 

Asthma 

Human 

Thing 

111 

Dream 

Mastocarcinoma 

Hoof 


root.p2p.com_telegram@ 

root. p2p. sic _hepatitis_type_C@ 

root. p2p. sic -Asthma© 

root . p2p.sic _Adult _Children@ 

root.p2p.sic_Alzheimer’s_Disease@ 

root. p2p. sic .Jacob .Disease© 

root. p2p. sic _Dealer@ 

root . p2p .sic _Mastocarcinoma@ 

root. p2p. sic _Hoof-and-Mouth-Disease@ 



2.3 Determining Target Agents Using Both Histories 

In order to determine the target agents to send a user query, an IR agent uses 
the contents of retrieved document files and two histories, Q/RDH and Q/SAH. 
Fig. 5 depicts an example how the target agents are found, where @ to © 
represent IR agents. For simplicity, we assume here that the IR agent does the 
job of an HM agent. Furthermore, to show the correspondence between a query 
and a retrieved document, we show the content files in Q/RDH. 

@ has two query entries in its Q/RDH. Both queries were sent by @ itself. 
This figure shows that @ sent query ’Note PC’ to © and got the retrieved 
results from ©. © recorded the query and @’s address into its Q/SAH. Since 
@ received the results from ©, ©’s address was recorded in the ‘from’ field 
of the same record as the query in @’s Q/RDH. In addition, since the content 
included in the results is the original of ©’s user, ©’s address is seen in the 
‘original’ field of the content. In the same way, @ also sent query ‘Wireless LAN’ 
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Q/RDH 



query: Note PC. 

from 

title : 
text : ... 
original :C 
range: ALL 



H Content 



query: Wireless LAN 
from :D 
title : ... 
text : ... 
originahE" 
range: ALL 
Q/SAH 



..... ( C 



Q/RDH 



Q/SAH 



'query :Note PC 
frdm :A 



query.,;Mini Note PC 
from :B 



A sends query 
“Wireless LAN” 
to both D and E 



A E 



C sends 
query ’’Note 
PC” to both 
A and B. 



Fig. 5. Example to find target IR agents using two histories and content file. A to E 
in circles represent IR agents’ names 

to @, @ returned retrieved documents to it, and @’s address was recorded into 
the ‘from’ field of the same record as query ‘Wireless LAN’ in @’s Q/RDH. 
Since the documents include a content created by @’s user, ©’s address is seen 
in the ‘original’ field of the content. 

After getting these histories, if © sends another query which is similar to 
‘Wireless LAN’, say ‘LAN’, @ not only can find @ in a ‘from’ field of Q/RDH, 
but also find © from an ‘original’ field of the content file by calculating a simi- 
larity between the query and the content file. Accordingly © sends the query to 
both @ and ©. 

The figure also shows that © received query ‘Mini Note PC’ from @, and 
both the query and ®’s address were recorded into the Q/SAH. Even if © has 
not sent a query, it can find information related to the queries it received using 
its Q/SAH. Therefore when © sends a query, say ‘Note PC’, it will find @ and 
® with the Q/SAH and can consequently send the query to them. 

2.4 The Effect of Both Histories 

As mentioned in the previous section, both Q/RDH and Q/SAH help to find 
target agents to send a query to. If an IR agent can find a sufficient number of 
agents, no ‘query multicasting’ is carried out. Both histories, consequently, help 
to reduce communication loads between agents. 

The user’s positive or negative judgments concerning the retrieved results 
could be embedded into them in Q/RDH. These user evaluations are expected 
to be useful for finding target agents which will return relevant information, cre- 
ating a collaborative filtering effect. As a user creates more information, his/her 
IR agent can return the retrieved results to more queries. Such an IR agent con- 
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sequently receives more queries from other agents. Thus, the agent accumulates 
more information sources comprised of pairs of a query and a sender agent’s 
address in its Q/SAH. That leads to the emergence of a ‘give and take’ effect. 

2.5 Overview of Multi-agent Kodama 

The ACP2P method was implemented with Multi-Agent Kodama (Kyushu uni- 
versity Open & Distributed Autonomous Multi- Agent) [29]. Kodama comprises 
hierarchical structured agent communities based on a portal-agent model. A 
portal agent(PA) is the representative of all member agents in a community 
and allows the community to be treated as one normal agent outside the com- 
munity. A PA has its role limited in a community, and the PA itself may be 
managed by another high-level portal agent. A PA manages all member agents 
in a community and can multicast a message to them. Any member agent in a 
community can ask the PA to multicast its message. All agents form a logical 
world which is completely separated from the physical world consisting of agent 
host machines. That means agents are not network-aware, but are organized 
and located by their places in the logical world. This model is realized with 
the agent middle- ware called Agent Communication Zone(ACZ for short). ACZ 
is primarily designed to act as a bridge between distributed physical networks, 
creating an agent-friendly communication infrastructure on which agents can be 
organized in a hierarchical fashion more easily and freely. ACZ is also designed 
to realize a peer-to-peer communication between agents. 

A Kodama agent consists of a kernel unit and an application unit. The kernel 
unit comprises the common basic modules shared by all Kodama agents, such as 
the community contactor or message interpreter. The application unit comprises 
a set of plug-in modules, each of which is used for describing and realizing a 
specialized or original function of agents. For more details, please see [29]. 

3 Experiments 

3.1 Preliminaries 

We used the Web pages of Yahoo! JAPAN[26] for the experiments. The Web 
pages used are broadly divided into five categories: animals, sports, computers, 
medicine, and finance. Each of them consists of 20 smaller categories, which are 
selected in descending order of the number of Web pages recorded in a category. 
An IR agent is assigned to each selected category, and thus 100 IR agents are 
created and activated in the experiments. A category name is used as the name 
of an IR agent, and the Web pages in the category are used as the original 
documents of the agent. All agents are realized by implementing their functions 
in plug-in modules of Kodama’s application unit. 

Each IR agent sends 10 queries, which all belong to either query set QL=1 or 
QL=2. QL=1 and QL=2 consist of 10 queries, whose query length is one and two, 
respectively, where query length means the number of terms in a query. When 
using queries belonging to QL=1, 10 nouns are extracted from every category 
assigned to each IR agent in descending order of their frequency of occurrence in 
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the category. Each of the nouns is used as a query of the IR agent. When using 
those belonging to QL=2, 5 nouns are extracted, and the combinations of the 
extracted 5 nouns taken in pairs create 10 queries. 

All IR agents were assigned to the same community for simplicity. 

We conducted experiments to show how two histories help to reduce commu- 
nication loads between agents looking for information relevant to a query, and 
how Q/SAH helps in searching for new information sources. To perform the ex- 
periments, we compared three methods : 1) ACP2P with a Q/SAH(wQ/SAH for 
short), 2)ACP2P without a Q/SAH(woQ/SAH for short), and 3) Simple method 
always employing a ‘multicast’ technique (MulCST for short). 

3.2 Similarity Measure for Information Relevant to a Query 

In order to find the requested number of target agents to be sent a query, we cal- 
culate Score(query,t-agent), which returns the similarity value between query 
query and target agent t-agent, with equation (1); Score(query, t-agent) be- 
comes higher if t-agent sends a greater number of similar queries and returns 
more documents related to query. After calculating Score(query, t-agent) for 
each IR agent t -agent in the Content file and both histories : Q/RDH and 
Q/SAH, the requested number (RN) of target agents will be selected in the 
descending order of Score{query , t-agent) , which value should be more than 0. 
Whenever the RN of agents is not found, the ’query multicasting’ technique will 
be employed by a portal agent. At that time, all answers will be returned to the 
portal agent. 

If a target IR agent finds information relevant to query from its Content 
files, it returns a ’YES’ message, otherwise a ‘NO’ message as mentioned in 
section 2.1. The judgment as to whether or not a document is relevant to a 
query is made according to the criterion of Boolean AND matching, that is, if 
the document includes the conjunctions of all terms in query , it will be judged 
relevant, otherwise irrelevant. 

k 

S cor e(query, t-agent) = cos (query, ghdj) 

i = 1 

m n 

+ ^(cos (query, qh sai ) + <p(i)) + ^ Sim d (query , doci) (1) 

i— 1 »= 1 

{ 6 if qh sai is the query sent by other IR 
agent directly. 

0 otherwise 

In equation (1), query consists of w ±, ..., w m , where Wi (1 < i < m) is a 
term in query. qh d and qh sa represent a query in a record of Q/RDH and 
Q/SAH, respectively. The first term )TA =1 cos(query, qhdi) returns the total 
score of the similarities between query and each of k number of queries sent 
to t-agent. The second term J/™ 1 (cos(< 7 uery, q/i sai ) + ip(i)) represents the 
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score between query and qh sail which is the i th of m queries sent by t agent 
in Q/SAH. ip(i) is a weight to consider the importance of ‘direct sending of a 
query.’ If qh sai is sent directly by t^agent, 6 is added to the score. The last term 
Y^i= i Sinid{query, doci) is the total score of similarities between query and 
each of n documents originally created or just owned by the user of t-agent. 
Sirrid{query, doc) represents the similarity between query and the content of re- 
trieved document doc. It is calculated with the following equation, which is a 
simplified form of BM15[20]. 



Simd(query, doc ) 



tfi 

h ^ +1 



Where tfi represents the frequency of occurrence of Wi in doc. 

The reason why we did not consider a “inverse document frequency” factor 
on the equation is based on preliminary experimental results. 



3.3 Experimental Results 

First, in order to decide the value of 6 in qh sai of equation 1, and to broadly 
figure out whether or not Q/SAH helps in looking for new information sources 
and in creating virtual communities, we compared three methods : woQ/SAH, 
wQ/SAH(6 = 0) and wQ/SAH(6 = 0.1). wQ/SAH(6 = 0) represents the method 
wQ/SAH with 6=0 and wQ/SAH(6 = 0.1) is the method wQ/SAH with 6=0.1. 
The results are shown in table 4, and show that both methods adopting Q/SAH 
are better than woQ/SAH. In addition, wQ/SAH(6 = 0.1) was slightly better 
than wQ/SAH(6 = 0) from the points of view of reducing the number of messages 
exchanged, increasing the number of achieved results and the number of agents 
making bidirectional communications. From these results, we decided to adopt 
wQ/SAH(6 = 0.1) as wQ/SAH. 



Table 4. The effect of 6 for wQ/SAH 



RN=5 


woQ/SAH 


wQ/SAH(6 = 0) 


wQ/SAH(6 = 0.1) 


# of Messages 


46112 


37361 


30050 


# of retrieved results 


9223 


13673 


14014 


# of One way agents 


307 


245 


206 


# of pairs of Bidirectional agents 


144 


272 


326 


The retrieval failure ratio 


10.79% 


8.44% 


11.36% 



Next, to show how much wQ/SAH works for reducing communication loads, 
we investigated the change of the average number of messages exchanged by 
each IR agent for every query input. The experiments were conducted with two 
query sets: QL=1 and QL=2, on which tests with 4 different requested numbers 
: RN=3, 5, 7 and 10 were performed. The results are shown in Fig. 6. In both 
cases, the average number of messages exchanged by each IR agent is reduced 
for every additional query input, and increases as RN does. 
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Fig. 6. The average number of messages exchanged by each IR agent for each query 
input, every query belongs to QL=l(left) or to QL=2(right) 



Further, for both QL=1 and QL=2, we compared the three methods: 
wQ/SAH, woQ/SAH and MulCST. RN was set to 10. The results are shown 
in Fig. 7. In both cases, the number of exchanged messages in MulCST did not 
change for every query input, while that for both wQ/SAH and woQ/SAH was 
reduced. In addition, wQ/SAH had better performance than woQ/SAH. 




Fig. 7. The average number of messages exchanged by each IR agent for each query 
input, where QL=1 is the left, and QL=2 the right. RN=10 in both cases 



We also compared three methods on the average number of documents ac- 
quired by each IR agent. The results are shown in table 5. Except for the case 
of RN=3 of QL=2, there was little difference between wQ/SAH and MulCST. 

Fig. 8 compares the three methods on the content acquisition efficiency, that 
is, the average number of acquired documents per one exchanged message in the 
cases of RN=3, 5, 7 and 10, for QL=1 and QL=2. For both query sets, wQ/SAH 
had the best performance, which was for RN=7 in QL=1 and RN=10 in QL=2. 

Further, we investigated the failure ratio caused by wQ/SAH and woQ/SAH 
because making use of query histories was just a good heuristic and might have 
failed to select appropriate target agents. The experiment was carried out with 
QL=2 because in the case of QL=1, no failure will occur since every query sent 
by each IR agent is different and the query is created from the original documents 
of the agent. The experimental results are shown in Fig. 9. 
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Table 5. Comparison on average number of acquired documents when query length is 
1 (left) and 2 (right) 



QL=1, RN= 


3 


5 


7 


10 


QL=2, RN= 


3 


5 


7 


10 


wQ/SAH 


269.1 


385.3 


443.0 


497.6 


wQ/SAH 


54.9 


126.3 


178.9 


226.8 


woQ/SAH 


258.8 


331.6 


424.9 


476.4 


woQ/SAH 


54.8 


96.8 


150.0 


208.3 


MulCST 


233.8 


366.4 


421.3 


487.0 


MulCST 


85.3 


148.4 


191.0 


232.6 





The RN of retrieved results 
I— wQ/SAH - woQ/SAH ^MulCST 



Fig. 8. Comparison of Content Acquisition Efficiency 

As the value of RN increases, the failure ratio decreases. This is because, as 
the value of RN increases, the number of acquired documents also increases as 
shown in table 5, and thus the number of information sources also increases. 
Consequently, it becomes easier to select target agents having information rele- 
vant to a query. As a baseline, we investigated the ratio of the number of agents 
returning a ’YES’ message to each query, to the total number of IR agents. It 
was about 20%. That means when target agents are randomly selected, about 
80% retrieval failure can occur. Considering that the retrieval failure ratio for 
both wQ/SAH and woQ/SAH was less than 20%, we can say that making use 
of two histories is effective in finding information relevant to user queries. 

Lastly, we conducted an experiment to show how the number of agents ex- 
changing query messages together increases, comparing wQ/SAH and woQ/SAH. 

QL=2 




| wQ/SAH woQ/SAH] 



Fig. 9. Failure Ratio for each RNof target agents, QL=2 
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The results are shown in Fig. 10. In the case of wQ/SAH, the number of pairs of 
agents making bidirectional communication is much greater than that of those 
making one way communication. On the other hand, woQ/SAH shows the oppo- 
site tendency. These results show that Q/SAH facilitates bidirectional commu- 
nications and then creates virtual agent communities where agents exchanging 
similar queries, i.e. , having the same interests, stay together. 
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Fig. 10. The change of number of pairs of agents making bi-directional or One-way 
communication, for QL=2 



4 Related Work 

There is lots of work related to the topics touched on in this paper, such as 
distributed information retrieval(DIR), P2P file searching, collaborative filtering 
and so forth. DIR selects some IR systems to send a query, aggregates the results 
returned by the selected IR systems, and presents them to a user. Before selecting 
the IR systems to be sent a query, the resource description of each IR system 
is often created[3]. In the ACP2P method, Q/RDH incrementally creates an 
effect similar to resource description, and furthermore, Q/SAH works as a good 
heuristic like collaborative filtering in finding relevant information. 

A lot of P2P file searching systems such as Freenet[5], Chord[24], Gnutella[8], 
Napster[17] and pSearch[25] have been proposed. Freenet and Chord are carried 
out in a pure P2P computing architecture. They neither employ ’broadcast’ tech- 
niques like Gnutella, nor have a centralized server machine like Napster. Freenet 
provides information sharing and information finding functions among anony- 
mously distributed nodes. Although Chord does not provide anonymity of nodes, 
it has an efficient protocol for looking up nodes. Their node searching strategies 
are conducted according to keywords attached to the information of the nodes. 
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On the other hand, the ACP2P method makes use of the content information of 
documents, and two histories: Q/RDH and Q/SAH to search for target agents 
with relevant information. In particular, Q/SAH provides similar effects to link 
analysis like PageRank[2] or HITs algorithm[13] and makes a natural collab- 
orative filtering effect emerge. pSearch[25] realizes a semantic overlay network 
on physical nodes in a content-addressable network(CAN) [19] by distributing 
document indices based on document semantics, which are generated by Latent 
Semantic Indexing(LSI)[6]. The search cost for a given query is thereby reduced, 
since the indices of semantically related documents are likely to be co-located in 
the network [25]. However, since LSI requires a document-term matrix, pSearclr 
initially needs to collect all documents from every node. 

I-Gaia[7] is an application layer for information processing in the DIET ar- 
chitecture, which is a Multi-Agent System development platform. It is formed 
of three types of agents: s-infocytes(SI), m-infocytes(MI) and t-infocytes(TI). It 
defined informaion-pull and push tasks with Reuters text-classification corpus 
and showed their results. For both tasks, queries sent by SI and documents pub- 
lished by MI reach Mis and Sis through TI, respectively. Thus, adequate routing 
between Sis and Mis is learned by TI. On the other hand, the ACP2P method 
does not rely on a special agent like TI for sending queries or publishing docu- 
ments, but each IR agent directly communicates with other IR agents based on 
a peer-to-peer communicating method. 

Although lots of work on the field of Collaborative Filtering (e.g. [9], [18], [23], [1], 
[11], [15], [21]) has been done, most of it assumes the server-client computational 
model and needs a procedure to collect all data from other nodes explicitly. 
The ACP2P method takes a distributed data management method with agent 
communities based on a P2P computing architecture, and makes a natural col- 
laborative filtering effect emerge, with both Q/RDH and Q/SAH. 



5 Conclusion and Future Work 

We discussed an agent community based information retrieval method, called 
the ACP2P method, which used the content of retrieved document files and two 
histories: Q/RDH and Q/SAH to find target agents to be sent a query. The 
method was implemented with Multi- Agent System Kodama. 

We conducted several experiments to show whether or not two histories 
helped to reduce communication loads between agents in searching for informa- 
tion relevant to a query, and whether or not Q/SAH helped in looking up new 
information sources. The experimental results showed the efficiency of ACP2P 
method and the usefulness of two histories for looking up new information 
sources. We also investigated and confirmed that the number of agents exchang- 
ing query messages together was increased by Q/SAH. 

We are currently investigating how to measure the accuracy of or to rank re- 
trieved results, and considering how we can make use of user feedback embedded 
into the results. Developing an effective method for creating hierarchical agent 
communities to allocate agents to at the initial stage is future work. 
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Abstract. We are investigating how to provide intelligent, pervasive 
support of group of people within so-called “smart environments”. Our 
current main assumption, based on literature in psychology and organi- 
zational studies, is that a group performs some complex, routine task as 
a structured activity, that is, by following some protocols that allow its 
members to coordinate and share a common understanding about the 
current progress towards the group’s goal and the roles currently played 
by each member. If this is the case, a condition to provide support to 
a group activity by artificial agents is to share the same understanding. 
To this end, we have identified two initial goals: first, being able to un- 
derstand if a group activity is progressing with respect to its expected 
evolution, by analyzing what is happening within the smart environment; 
second, recognizing what are the social roles of the group members, tak- 
ing in mind that these are not necessarily pre-assigned and may change 
in time. This paper sketches a preliminary approach to these issues and 
a computational model for an overhearer agent. We suggest a prelimi- 
nary set of rules for conversation analysis and social role recognition, and 
validate them against the simple case of implicit organizations, which - 
being artificial - follow well-known protocols. 



1 Introduction 

An active environment is a physical space that can sense and respond appro- 
priately to the people and activities taking place within it [1], These kinds of 
environments should provide assistance to the users in their physical or cogni- 
tive tasks; in particular, when the users are involved in a group activity, it is 
essential for the success of their work that the environment provides such assis- 
tance. Our long-term objective is intelligent, pervasive support of people within 
“smart environments”, motivated also by the availability of new sophisticated 
technologies, such as user interfaces and tools which enhance the possibility of 
human interaction, during the activity, with a combination of different input 
modalities (gesture, voice, etc.). Within the PEACH project [2] we are focus- 
ing on a interactive museum. This is a form of active environment, and can be 
seen as a large scale multi-user, multi-media, multi-modal system. In the case 



M. Klusch et al. (Eds.): CIA 2004, LNAI 3191, pp. 47-61, 2004. 
(c) Springer- Verlag Berlin Heidelberg 2004 




48 



S. Rossi and P. Busetta 



of PEACH, museum visitors are provided with information about exhibits; this 
may be drawn from a variety of information sources and media types (museum 
server, online remote servers, etc.), and presented by a variety of clients (e.g., 
hand-held devices such PDAs, kiosks, wall screens, and so on). 

We are currently working towards exploiting channeled multicast [3] for in- 
telligent support of groups of people, rather than individuals. In the museum 
setting, this is a particularly sensitive issue, since group visits are common and 
historical buildings must accommodate a number of simultaneous users. Our 
current main assumption, based on literature in psychology and organizational 
studies, is that a group performs some complex, routine task (such as planning 
a visit following the group’s preferences) as a structured activity, that is, by fol- 
lowing some protocols that allow its members to coordinate and share a common 
understanding about the current progress towards the group goal and the roles 
currently played by each member. This assumption seems reasonable especially 
when we consider professional routine activities such as daily briefings. If this is 
the case, a condition to provide support to a group activity by artificial agents 
is to share the same understanding. 

As first step towards this objective, for this initial work we concentrate on 
multi-agent systems, which - being artificial - follow well-known protocols. We 
have identified two initial goals for our own work. First, being able to under- 
stand if a group activity is progressing with respect to its expected evolution, by 
analyzing what is happening within a smart environment. In this sense, we want 
to find a way to monitor (and possibly recognize) a conversation that reaches 
certain known states, via overhearing all the messages exchanged through broad- 
cast communication. Second, finding a way to recognize what are the social roles 
of the group members, taking in mind that these are not necessarily pre-assigned 
and may change in time. This is important non only for a single conversation 
but for the evolution of their (long term) interaction as a whole. In fact, social 
roles define the interdependencies among members and set expectations for the 
behaviors of the members. 

In the current literature on multi-agent systems, the role of overhearing and of 
monitoring on distributed systems is gaining attention, as demonstrated by some 
recent works, e.g. [4, 5,6,7]. In particular, in [4] the authors provide a formal 
model of the overhearing process in order to recognize conversation protocols. 
These are formalized as Finite State Machines where the conversation states are 
defined by the Cartesian product of each single agent state; a transition function 
is defined on all the possible “legal” states. They introduce also the concept of 
recognition of conversation roles, but such roles are just the ones defined by the 
protocol itself. 

As in [4], this paper sketches a computational model for an overlrearer agent, 
suggests a preliminary set of rules for conversation analysis and social role recog- 
nition, starting not form the point of view of each single agent, but from the 
point of view of the entire organization of agents, formally represented with 
formulas in the JIT [8] (Sec. 2). We validates them against the simple case of 
implicit organizations (Sec. 3). 
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2 Representing Group Activities Through JIT 

To recognize or to model simple patterns of interaction (i.e. natural policies or 
protocols), one of the most obvious ways to proceed is, as we see from the cur- 
rent literature [4,5], to model them as Finite States Machines. A finite state 
machine is an abstract machine consisting of a set of states, a set of input events 
and a state transition function. Recently, Kumar et al. [9] and Chopra et Singh 
[10] pointed out that a state machine representation of a protocol has many 
limitations. The main criticism is that, by labelling arcs with communicative 
actions, these actions are fixed, so in open multi-agent systems, where agents 
are autonomous and heterogeneous, such agents cannot handle exceptions and 
opportunities. Moreover, there can be several actions (communicative and non- 
communicative) that can lead to the same result [9]. This fact is of a great 
importance in our long-term objective of recognizing interactions among human 
beings, where we cannot predict all the possible evolutions. Moreover, assuming 
that such formalization is possible for humans, we foresee that this kind of repre- 
sentation lead us to a huge number of possible states, because the number of the 
possible interactions grows exponentially with the number of group members. 
Finally, we can be also in a situation where we do not exactly know who such 
members and so we cannot describe the states of the group as a product of their 
finite state machines. 

In the representation of conversation protocols made by [9], instead of spec- 
ifying the state transitions, the authors specify just a partially ordered set of 
states (called landmarks) using the Joint Intention Theory [8]. JIT was born as 
a follow-up of a formalization of the theory of speech acts [11] and is expressed 
as a modal language with connectives of the first order logic and operators for 
propositional attitudes. The basic idea behind the landmark representation is 
that since a protocol is used to perform a certain task or to bring about a cer- 
tain state of the world, one can identify the states that are brought about during 
the execution of the protocol. In addition, these states can be intended as the 
sub-goals that the group has to achieve in order to reach the global goal. So in 
conversation protocols, states can be characterized by a conjunction of proposi- 
tions that are true in the state represented by the landmark. As landmarks in 
landmark expressions can represent the ’’way-points” that must be followed to 
successfully execute a protocol, we can think of a way to monitor an interaction 
that reaches certain known states, common for a given type of interaction. 

The authors in [9] also show that once that the representation of the fam- 
ily of protocols through landmarks is given, it is possible to derive the specific 
protocols. For what concerns us, we do not expect that from our landmark ap- 
proach we can directly derive the specification of a protocol. What we want is 
not to be able to express a family of protocols, but, on the contrary, to be able to 
monitor organization of agents that use such protocols. This can be done having 
the states, which the organization must follow, as available information (land- 
marks); such landmarks represent the states of the organization expressed as 
beliefs and intentions of groups. Our objective is not to evaluate the correctness 
of the protocol (as in the classical use), but to monitor the development of the 
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group state. Specifically, we want to define a general set of social conventions 
and interaction policies, formalized with the Joint Intention Theory [8,12], for 
establishing and recognizing an interaction policy among human groups, where 
ordering of landmarks represents the state that the group presumably will follow 
and actions to reach such state can be chosen dynamically (as in human inter- 
action). Of course, we can’t write JIT-formulas for each single member if the 
number of such agent is not known a priori, but, through a group extension of 
a few basic concepts, as done in [13], and introducing quantifiers, is possible to 
represent dynamic groups in a formal language and to reason about the whole 
group. Given these extensions of the JIT, using formulas for recognition, we have 
to deal with some issues such as: 

Different Interactions: For going to the current landmark to the next land- 
mark, the group can exchange different kind of messages. For example in the 
Contract-Net Protocol (as formalized by Kumar et al., see [9]) the transition 
to the final state, where the job cannot be done or the escape condition is 
true, can be reached by exchanging different messages. As we said above, 
different actions can lead to the same result. 

Multiple Messages ( Case 1 ) when agents are known: For going to the cur- 
rent state to the next state, the group needs of exchanging n messages from 
n different agents (see again [9] , in the Contract-Net Protocol the transition 
to the final state needs the exchange of two messages from the two agent 
involved). So, we can express interaction protocols in a compact way. 
Selection of the State: In computing the next expected state, one message 
can be compatible with more than one next state. This means that, when 
dealing with a flexible and fault tolerant protocol, actions can be ambiguous. 
Multiple Messages (Case 2) when agents are known: For going to the cur- 
rent state to the next state, the group needs of exchanging n messages from 
m different agents with (to < n). This is the case, for example, of an Auc- 
tion in multicast channel (where everybody can overhear everybody else’s 
messages), where all bidders can listen all the bids, so they can change their 
own bids and bid again (or wait until they understand how to bid to win). 
This is exactly the case of the following: 

Multiple Messaging when agents are unknown: For going to the current state 
to the next state, the group needs of exchanging a number of messages that 
is unknown because the number of agent involved is unknown. 

Another key aspect to take into account is that people in a group may play 
various roles, and in this representation the concept of social role is still out. Of 
course, once that interaction protocols are described using the JIT, the natural 
way to formalize the roles involved in the interaction is by describing their be- 
havior according to their Joint Commitment towards the group [8]. In traditional 
use, roles are typically static in the sense that membership of a given role is es- 
tablished early and rarely changed [14]. In CSCW applications, it is helpful to 
try to supply roles directly, but group dynamics can be difficult to establish, and 
individual activities can vary from group to group, from one context to another 
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[15] . Having a topological view of the different roles within an application and of 
the relations between these roles is crucial if we want to manage multi-party in- 
teraction and to define policies of interaction. In this sense, in order to recognize 
social roles we need of some additional information as defined in the next section. 

2.1 Recognition of Joint Activities 

When agents are members of a group that share a common goal, they are engaged 
in interactive communicative social roles. These roles can be viewed as expecta- 
tions for the individual, just as norms are expectations for a group [16]. There 
can be different types of roles, e.g. for task accomplishment, socio-emotional or 
just communication roles. In our case the roles that are associated to the agent 
in the recognition process are both social and conversation roles and depend on 
the current type of interaction (i.e. protocols). Therefore, in order to recognize 
such roles we must have a model of the interaction of our group of agents and 
we must know how these roles are related to such interaction. 

In our scenario, we assume that an Overhearer wants to recognize social roles 
in order to pro-actively provide suggestions, influence behaviors [17] and recov- 
ery from failures. As we said in the introduction, this work is a first step towards 
our objective, so we focus on artificial multi-agent systems where communica- 
tion happens via observable multicast communication, which enables overhearing 
[13] . Thanks to overhearing, teamwork can be monitored so that team-members 
can coordinate their actions and plans with team-mates, and help team-mates 
to cooperate without interference [7,18]. Generally speaking, teams differ from 
small groups in the sense that a team is formed in order to achieve a specific 
goal; roles within a team are just task-dependent and not social [16]. In this 
work, we use the word “team” or “organization” as synonyms for a group with 
a shared goal but that has not only task-dependent roles. 

For this initial work, we designed a simple language that allows the definition 
of the interaction knowledge, intended as two types of rules. The first one asso- 
ciates a message with the current state of the group (as the sender perceives it) 
and the intended effects of such message, and the second associates a message 
again with a current state of the group, but also with a social role. In the formal- 
ization of the possible messages, preconditions and effects are described explicitly 
as JIT-formulas; Preconditions and effects refer to the organization as a whole. 
As we will see in Sec. 2.2, consistency checking is based on matching of precondi- 
tions of messages with landmark states and the computation of the next state is 
based on the same matching done on the effects. Observe that preconditions and 
effects can be part of a landmark (landmarks are conjunctions of formulas), so the 
same rules can be applied in different landmarks and may cause different results. 

In our current work, more than one message rule may be associated to each 
type of message (i.e. to each type of performative with a defined sender and re- 
ceiver role), but, preconditions of each message-rule have to be different. This is 
to say, the same message can have different effects, but only if its preconditions 
are different (i.e., it is exchanged in different contexts). Preconditions may also 
be not specified intending that they are always satisfied. In a landmark repre- 
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sentation, landmarks can be defined by a conjunction of proposition; in the same 
way, precondition and effects in message-rules are conjunction of propositions. 
We reason about message-rules by means of a BDI [19] engine, based on agentS- 
peak(L) [20] and a reasoner implemented in Prolog. Thus, message rules can be 
seen as a plan library, overheard messages are trigger events and preconditions 
of the rules have to match with the current beliefs of the overhearer agent. 

The grammar of the message-rules is expressed by the following BNF: 

<MessageRule> := <Message> 

"Preconditions" "=" JIT-Formulas 
"Effects" "=" JIT-Formulas 

<Message> ::= <Performative>" (" <Sender> <Receiver> <Content> ")" 

<Perf ormative> ::= "REQUEST" I "QUERY" I "DONE" I "INFORM" I "ASSERT" 
<Sender> ::= <AgentIdVariable> 

<AgentIdVariable> ::= STRING 

<Receiver> ::= " ' " <Role> I <AgentIdVariable> 

<Role> ::= STRING 

<Content> ::= <ContentVariable> I [Some specific communication 
language schema with embedded variables] 

<ContentVariable> ::= STRING 

For example, the following rule: 

REQUEST (Requester, ‘ExampleRole ’ , <Job> p </Job>) 

Preconditions= ->(DONE 1 ExampleRole' p) A 

-i (JPG ‘ExampleRole' ( DONE ‘ ExampleRole ’ p)) 

Effects= ( JPG ‘ ExampleRole' ((DONE ‘ ExampleRole' p)A(BEL Requester p))) 

says that a request sent by a Requester to “ExampleRole” whose content is an 
XML document of type “Job” requires, as its precondition, that the requested 
job has not been done yet and has the effect of creating a joint persistent goal 
for the “ExampleRole” group of agents. 

The second set of rules gives us the possibility to make inferences about social 
roles of the members of the organization. This kind of aspect is not captured 
using a JIT reasoner only. Rules are formed by a message template to which it 
is associated a set of possible states and a possible role. Observe that even if the 
roles are related to a single agent, in particular to the sender of the message, 
the possible states do not refer to the single agent but to the organization under 
monitoring. The grammar for the roles is the following: 

<SRoleRule> ::= <Message> 

"LegalLandmarks" "=" <Landmarks> 

“->” "SocialRole" "=" <SRoles> 

<Message> ::= [see previous figure] 

<Landmarks> : : = <Landmark> " , " <Landmarks> I <empty> 

<Landmark> ::= <LABEL> "=" JIT-Formulas 
<LABEL> := STRING 

<SRoles> ::= <SRole> <SRoles> I <empty> 

<SRole> ::= STRING 
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(1) message from the channel 

(2) call to the agent/role recognition process 

(3) check for the consistency of the message 

(4) consistent messages are forwaed to the group-level state recognition 

(5) call to the role recognition function 



Fig. 1 . Computational model for the overhearer 



For example, the following rule: 

INFORM (Sender, Receiver, Context) 

LegalLandmarks = WORKING 

-> SocialRole = ACTIVE-PARTICIPANT 

says that an INFORM sent when the group activity is at a landmark labelled 
as “WORKING” implies that its sender is an active participant to its group’s 
activity. In Sec. 3.1, we show some more examples and discuss how this rules are 
applied to the case of implicit organizations. 

2.2 A Computational Model for the Overhearer 

The general computational model for an overlrearer, able to interpret the 
rules presented above, is composed of two main concurrent processes (see 
Fig. 1). 

We distinguish a main overhearing module from a module dedicated to the 
recognition of the current state (landmark) of the group and the possible 
next states through the application of the message-rules (for brevity called 
overhearing and group-level, respectively). Goals of overhearing are: (1) 
to handle a list of the agent within the organization; (2) to assign to each agent 
and to modify, during the interaction, a particular social role and (3) to handle 
all the tasks of monitoring the behavior of such agents in accordance with their 
roles. This process is achieved by using two external functions: is-consistent 
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and get-role. The first function checks the consistency of a message with the 
current state of the organization and with the possible next states. This process, 
as described below, is made by applying the message-rules. If the message is 
consistent, it is forwarded to the group-level. The second function recognizes 
the role played by the agent by applying social-rules. Both functions need to 
know the current state of the organization. This value (as well as the possible 
next states) is maintained in a global variable; in this way it can be read by 
these functions and written by the group-level. We assume that any problem 
of concurrency is handled by the implementation. 

In this preliminary work, the recognition of the type of interaction consists 
only of recognizing the transition from one state of the group to another state. 
We assume that is possible to understand which kind of interaction is going on (as 
done by [4]), and so we provide the overlrearer with the landmark representation 
of each of them. Of course we foresee that this kind of approach (i.e. purely 
symbolical) lead to a high computational complexity, so we plan to extend our 
representation through a hybrid statistical/symbolic approach in future works. 
We are currently testing the algorithms on the example of Section 3 and results 
will be published in a forthcoming paper. 

The Algorithm 1.1 specifies the initialization of global variables and the 
interaction with the function Agent-Level, that has the task of updating the 
list of the agents and their associated roles. The algorithm starts also some 
role-dependent monitoring activities, which are out of the scope of this 
paper. 

Algorithm 1.1 Overhearing 

> AGENTS = {(Ai, srole)\Ai £ Agent-identifier, srole £ SRoles}: known agent/ social-role 
couples 

> Si, Si+i = {s_j} = Current and expected states of the organization, respectively 

1 Si <- INACTIVE; 

> Initialization of the variable s^ to a void state 

2 repeat 

3 rrii <— GET-MESS AGE(); 

4 AGENTS <- AGENT-LEVEL(mf, AGENTS); 

t> updating the list of Agent to Monitor 

5 DO-MONITORING(AGENTS) ; 

> start the activity of monitoring 

6 until STOP-EXECUTION. 

Agent-Level (see Algorithm 1.2) selects which agent to monitor, that is those 
for which it has been able to recognize the social role. This function takes as 
input an agent-list and a message and returns such list updated. Agent-Level 
calls another function to check the consistency of the message with the current 
state of the organization; if this message is consistent, the agent-level calls the 
function Get-Role (see Algorithm 1.3), that, on behalf of the current state of the 
organization and the social-rules, associates a specific social role to the sender 
agent. 
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Algorithm 1.2 Agent-Level(m, agents) 

> Si = Current state of the Organization 

> message mi = (PERFORM AT IV E (Sender, Receiver, Content)) 

> AGENTS = {(Ai, srole)\Ai £ Agent-identifier, srole £ SRoles}: known agent/ social-role 
couples 

1 Ai <— mi. sender-, 

2 if IS-CONSISTENT(A,m<); 

srole <— GET-ROLE(Ai, mi); 

AGENTS <- AGENTS + (Ai, srole); 

3 Return AGENTS. 

Algorithm 1.3 Get-Role(A;, mi) 

> Si = Current state of the Organization 

1 Return srole — APPLY-S-ROLE-RULES(si, nn). 

During the interaction a lot of possible inconsistencies may happen. The func- 
tion Is-Consistent takes as input an agent and its message and reads the cur- 
rent state of the organization and the next expected states from the global vari- 
ables (see Alg. 1.4). Then, it tries to apply the message rules (APPLY-MESSAGE- 
RULES). The latter function takes the effects of the message and tries to match 
them with all the possible legal landmarks, then returns all the landmarks 
which include such effects. At this point, the consistency is computed on the 
preconditions of the message in a similar way as the matching on the effects 
(CHECK-PRECOND). If the message results consistent with the organization, it is 
forwarded to the Group-Level, otherwise an error message is sent to the agent. 

Algorithm 1.4 Is-CONSiSTENT(Aj, to,;) 

> agent-identifier Ai 

> message mi 

> Si, Si+i = {sj} = Current and expected states of the organization, respectively 

1 IS-COMPATIBLE <- CHECK-PRECOND (mi, Si); 

2 Sm = {sm|sm £ landmarks} <— APPLY-MESSAGE-RULES(mi, Si); 

> Try to compute the next landmarks for the message 

3 if ((S m C 5 i+ i)A IS-COMPATIBLE) V CHECK-PRECOND (mi , S i+1 ) 

then STORE-MESS AGE(mi ) ; 

Return TRUE; 

4 else 

SEND-MESSAGE(INFORM( “Overhearer”, Ai,type-of-inconsistency)); 

Return FALSE. 

Obviously, the recognition of the states of the group depends on the interac- 
tions made by the single agents. However, for being able to recognize the state of 
the organization, besides the expectations on the next state given by the land- 
marks, we have to analyze all interactions made by the agents. The Algorithm 
in 1.5 can access a buffer that contains all the consistent messages forwarded by 
the function Is-Consistent. In this sense, the task of the Group-Level is to 
update the current state of the organization and to compute the expected next 
states. The process of selection of such states is done, as usual, by matching 
each single message effects with the legal landmarks. During this process we can 
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find situations in which such effects are compatible with more than one legal 
landmark. In this sense (before that the organization reaches the next state) we 
can have that next states are a set of landmarks. Obviously, this process will 
finish when a single next state is selected and all its JIT-formulas are valid. 

Algorithm 1.5 Group-Level 

> Si = Current state of the Organization 

> Si - |-i = {si+i} =■ set of next expected states of the Organization 

> buff er Of Messages = { mi\mi £ messages }: Buffer of consistent messages 

1 repeat 

2 S i+ 1 APPLY-MESSAGE-RULES(FIRST-OF({mf})); 

3 if (|5 i+ i| - 1)A VALID(5 i+ i) 

> the cardinality of the set is one and the organization has reached the next state 

Si <- FIRST-OF(Si + i) 

{Si+i} LEG AL-L ANDM ARK(«i ) ; 

4 until STOP-EXECUTION 

In this algorithm, when the set of the possible expected states contains just 
one state which is valid (i.e. its associated JIT formula is satisfied), such state 
is set as the current state. Of course, this kind of computation is not trivial. 
In our preliminary approach we don’t have a process to reason formally about 
JIT formulas; our method is based just on simple plan matching exploiting our 
BDI interpreter. But, even in the case of having the ability to fully reason on 
JIT, we cannot be sure that all the information needed is exchanged (or all 
the messages are overheard). In this sense to compute the function VALID we 
use the following assumption: let us assume that all the messages overheard are 
compatible with the same current state (such assumption is true for Group-Level 
because only the consistent messages are stored in the buffer); when, in the 
selection of the next state, we have a single next state and we overhear a message 
that is compatible with such state, we can assume that all the formulas in this 
state are valid. Of course this is a strong assumption, but we plan, in order to 
enforce it, to provide a process to reason about JIT formulas. 



3 A Case Study: Implicit Organizations 

In our previous works [13,21] we did some investigations on the problem of 
group communication via overhearing in the field of multi-agent systems to sup- 
port ambient intelligence. In simple terms, our approach consists in modeling 
the components of an active environment as agents, and having them communi- 
cating via multicast channels [3] rather than via point-to-point connections. We 
proposed a special form of team, called implicit organization [13], for the coor- 
dination of agents joining multicast channels to play a specific role. The word 
implicit highlights the facts that there is no need for a group formation phase 
(joining an organization is just a matter of tuning on a channel), and no name 
for it - the role and the channel uniquely identify the group, indeed. We adopt the 
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term organization from Tidlrar [22], to refer to teams where explicit command, 
control, and communication relationships are established among sub-teams. 

We choose implicit organizations as a case study for the approach presented 
above and for overhearing as a mechanism to achieve stability and consistency of 
the organizations. This is because group communication protocols used by Im- 
plicit Organizations take into account problems coming from unreliable channels, 
continuous changes in the number of agents, and strict real-time constraints. So, 
our objective is to explore advantages and disadvantages of an approach based 
on overhearing on unreliable group communication, in a situation where agents 
can come and go fairly quickly, their capabilities can change or evolve over time, 
and it is not necessarily known a-priori which agent can achieve a specific goal 
without first trying it out. Observe that third parties overhearing a channel, 
in accordance to [23], may help in making the interaction with implicit organi- 
zations much more robust - for instance, by detecting some message losses, or 
whether a goal-specific sub-team has been established. 

3.1 Monitoring Implicit Organizations 

An implicit organization is in charge of defining its own control policy, which 
means: (1) how a sub-team is formed within the organization in order to achieve 
a specific goal (i.e., serving a request); and, (2) how the sub-team decides to 
pursue its goal. For the latter objective, a coordination policy has to be estab- 
lished by the organization and used by sub-teams to decide who actually works 
toward achieving goals, and possibly to coordinate the agents when more than 
one is involved in the process. Whenever a new agent joins an environment, 
a protocol is run to negotiate the coordination policy of the organization the 
agent wants to be part of; the same happens when an agent leaves or whenever 
somebody detects an inconsistency due, for instance, to network partitioning 
or agent crashes, or simply to changes in the environment. In [13], we assumed 
that any request - by which we mean any REQUEST and QUERY, using the 
FIPA performatives [24] - generates a commitment by an implicit organization 
to perform the necessary actions and answer appropriately. The negotiation pro- 
tocol, also described in [21], has been designed to work as efficiently as possible. 
In summary, the negotiation protocol works in two phases: first, the set of pol- 
icy instances common to all agents is computed; then, one is selected. During 
the first phase of the negotiation, a group belief about the common policy in- 
stances is established. This can be easily achieved by having each agent sending 
information messages (INFORM) to the role on what it is able to support, and 
intersecting the contents of all received messages. The second phase consists in 
having one agent, called the ’’oracle”, selecting one policy among those common 
to everybody and notifying the organization of its decision, by means of a simple 
assertion message (ASSERT) sent to the role. 

In the context of a coordination policy negotiation, the possible objectives of 
an overlrearer are: 
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Avoiding Renegotiation. The protocol described in [13] requires that every 
time an agent enters or exits from a channel, a renegotiation of the policy is 
called. Obviously, there will be cases in which such process is unavoidable (see 
as an example the case in which the agent which exits has an important social 
role in the organization, or the agent who enters wants to assume the role of the 
“oracle”). In all the other cases the agents (“new Agent”) can make a query to 
the over hearer and adapt themselves to the running situation. 

Consistency Check. During the process of policy negotiation, as we said, the 
protocol is described by means of the states in which there are mutual beliefs 
about some particular predicates. The role of the overlrearer, in this case, can 
be of verifying the inconsistencies of some members of the organization and the 
integrity of mutual beliefs concerning supported policies, common policies, and 
the current policy. 

The current implementation of implicit organizations is formalized in [21] by 
means of the Joint Intention Theory [8] through a landmark based approach [9], 
as in Figure 2 where the landmark are: 

LI : -<(DONE pp) A -(JPG p ( DONE p p) 

L2 : -.{DONE pp) A (JPG p {{DONE p p) A {BEL r p))) 

L3 -.{DONE pp) A {MB p {DONE p p)) 

L4 : {MB p CurrentPolicyDecided) A {INTEND a Coord{Pi a r p)) + L2 
L5 : {JPG p CurrentPolicyDecided) t \-<{MB p {Current Policy Pi)) + L2 
L6 : {JPG p CurrentPolicyDecided) A {MB p {CommonPolicies II)) 

A {GOAL o {MB p {CurrentPolicy select From{o, II)))) + L2 
L7 : {MB p {CommonPolicies LI)) A {MB p {CurrentPolicy Pi)) + L2 




Fig. 2. Request and Policy Negotiation Protocol for an Organization 



The machine represented above starts with landmark LI, that says that there 
are no pending goal, while landmark L2 says that the requested goal has not 
been achieved yet, and that the implicit organization p has a joint persistent 
goal to achieve it and notify the requester r. The notification of the results to 
r is overheard by everybody in p, thus it establishes the mutual belief within 
the organization that satisfies its joint goal (landmark L3). Landmarks from L5 
to L7 represent the policy negotiation protocol and in these landmarks all the 
formulas in L2 are still valid. 

In the case of the Request Protocol, the roles can be formalized as requester 
agent, participant (i.e. each agent of the organization receive the request for the 
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job) and active-participant (i.e. the members of the organization which actively 
participate to the achievement of the task). Moreover, during the policy negoti- 
ation phase we can have different kind of social roles such as the oracle (i.e. the 
agent which is in charge to select the coordination policy). 

SRoles = REQUESTER I PARTICIPANT I ACTIVE-PARTICIPANT I NEWAGENT I 
INITIATOR | ORACLE 

Finally, for sake of simplicity, the possible states of the organization, repre- 
sented by the landmarks, can be expressed as: 

Landmarks = INACTIVE LI I WORKING L2 I COMPLETE L3 I COORDINATING L4 I 
P-UNKNOWN L5 I P-NEGOTIATING L6 I P-DECIDED L7 

For instance during the policy negotiation protocol some of the rules for the 
overlrearer are: 

REQUEST (Sender, Receiver, Job) INFORM (Sender, Receiver, Content) 

LegalLandmark = INACTIVE LegalLandmark = WORKING 

-> SocialRole = REQUESTER -> SocialRole = ACTIVE-PARTICIPANT 

REQUEST (Sender , Receiver , CUR_POLICY () ) INFORM (Sender , Receiver , CUR_POLICY () ) 
LegalLandmark = P-UNKNOWN LegalLandmark = P-NEGOTIATING 

-> SocialRole = INITIATOR -> SocialRole = ORACLE 

In the following message rules, we assume that the communication language 
is XML based, as it is the case in our LoudVoice platform: 

MESSAGGES : 

REQUEST (Requester, ‘ExampleRole ’ , <Job> p </Job>) 

Preconditions= -i( DONE ‘ExampleRole' p) 

Effects= ( JPG ‘ExampleRole' ((DONE ‘ ExampleRole ' p))) 

DONE (ExampleRoleAgent , Requester, <Job> p </Job>) 

Preconditions= (DONE ‘ ExampleRole ' p) 

Effects= (BEL Requester p) 



4 Conclusions and Future Directions 

In this paper we present a first attempt to formalize the process of monitoring 
group conversations and of recognizing social roles via overhearing. In particu- 
lar we based our monitoring process on group’s states expressed through Joint 
Intention Theory and a landmark based approach. As with software agents, the 
problem of monitoring group interactions or teamwork remains a key problem 
also when dealing with human groups. Analyzing human behavior and group 
interactions is becoming possible by the availability of new and sophisticated 
technologies, such user interfaces and tools than enhance human-computer in- 
terfaces with a combination of input modality (gesture, voice, and so on). This 
analysis is needed because, if a smart environment has to provide implicit and 
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pro-active services, it needs to monitor and understand human activity and 
human needs. This has to be accomplished by establishing a model of human 
activities, and the technological and environmental state in which human beings 
operate. Human society has developed a set of commonly understood conversa- 
tion protocols (social norms) for many common interactions [9]. Obviously, the 
application of this approach to human beings implies a higher uncertainty and is 
left to future works once that some statistical process of interaction recognition 
model will be formalized. 
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Abstract. The problem of encouraging trustworthy behavior in P2P 
online communities by managing peers’ reputations has drawn a lot of 
attention recently. However, most of the proposed solutions exhibit the 
following two problems: huge implementation overhead and unclear trust 
related model semantics. In this paper we show that a simple probabilis- 
tic technique, maximum likelihood estimation namely, can reduce these 
two problems substantially when employed as the feedback aggregation 
strategy. Thus, no complex exploration of the feedback is necessary. In- 
stead, simple, intuitive and efficient probabilistic estimation methods 
suffice. 



1 Introduction 

Recent empirical studies have shown that much of eBay’s commercial success 
can be attributed to its reputation mechanism (Feedback Forum) as a means 
of deterring dishonest behavior. Thus, [1] shows that “reputation profiles are 
predictive of future performance” , while [2] and [3] come up with the conclusion 
that Feedback Forum completely fulfills its promises: the positive feedback of 
the sellers increases their price, while the negative one reduces it. 

eBay’s Feedback Forum is just a well known example of reputation systems 
[4] as informal social mechanisms for encouraging trustworthy behavior in on- 
line communities. Their key presumptions are that the participants of an online 
community engage in repeated interactions and that the information about their 
past doings is informative of their future performance and as such will influence 
it. Thus, collecting, processing, and disseminating the feedback about the partic- 
ipants’ past behavior is expected to boost their trustworthiness. The mentioned 
eBay example confirms this expectation. 

A huge body of work has appeared recently on managing online reputations. 
(For a comprehensive overview see [5] for instance). In this paper we will be 
more specific and consider only P2P networks. We will first review the relevant 
literature (Section 3) and offer a view on how various P2P reputation manage- 
ment approaches contribute to building trust. As we will see, most of the them 
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suffer from the following two problems: huge implementation overhead and un- 
clear trust related model semantics. The main cause of the first problem lies 
in the necessity of aggregating the feedback about all peers in the network in 
order to assess the trustworthiness of a single peer, while the second problem is 
mainly caused by the counterintuitive feedback aggregation strategies resulting 
in the outputs that are hard to interpret. In this paper we show on two settings 
relevant for P2P communities that a simple probabilistic technique, maximum 
likelihood estimation namely, can reduce these two problems substantially when 
employed as the feedback aggregation strategy (Sections 4 and 5). Operating on 
a small fraction of the feedback available in the network, it lends itself to an effi- 
cient implementation. On the other hand, its outputs are probabilities of specific 
behaviors of the peers and as such have a clear and well founded interpretation. 
Finally, its ability to detect peers’ misbehavior is as strong as that of the best 
ones of the existing approaches. Thus, we conclude that no complex exploration 
of the feedback is necessary. Instead, simple and efficient probabilistic estimation 
methods suffice. 

Interestingly, the vast majority of the existing approaches (excluding game- 
theoretic works) evaluate their effectiveness assuming that the peers are associ- 
ated with probability distributions determining their performance. However, this 
assumption on the peer behavior is not explicitly built into the solution meth- 
ods they propose. On the other hand, we start with the same assumption and, 
instead of aggregating the feedback in an ad hoc manner, we just try to predict 
the peer probability distributions as well as we can. This is the key difference 
between our and the other solutions. 

However, the exact setting in which the technique can be successfully used 
rules out forming huge collusive groups among peers. Instead, it implies their 
independent acting. 



2 P2P Computational Models of Trust 

In the following we introduce a general view on P2P computational models of 
trust, broad enough to cover all specific works we are aware of. The purpose 
of the section is to describe in general the problem the decentralized trust and 
reputation management considers. Section 3 then describes particular classes of 
solutions of the problem. 

An underlying assumption of the computational models of trust is that the 
peers engage in bilateral interactions, which results in forming a directed weighted 
(trust) multigraph. Its node set coincides with the set of peers and the set of 
edges with the set of interactions between the peers. In a general form, the weight 
assigned to any edge consists of an ordered pair: a flag representing the context 
of the corresponding interaction and the interaction outcome as perceived by 
the source. The set of all interaction outcomes is assumed to be common knowl- 
edge among the peers. We assume at most two possible interaction contexts: (1) 
recommendations, when the destination node acts as a recommender of other 
nodes capable of performing a specific task or other recommenders, and (2) the 
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task performances themselves. We also assume that the latter must be present 
while the former is optional. 



path 1 




Fig. 1 . Computational model of trust 

Figure 1 presents an example. The way we should understand this figure 
is as follows. Node a had three interactions with node b : once node b acted 
as a recommender of other entities (flag r) and node o’ s contention with the 
recommendation was evaluated 0.8 and twice node b provided the service in 
question to node a (flag d ) and a’s evaluations of the service provisions were 1 
and 0.9 respectively. 

The core of any computational model of trust is in the answer to the following 
question: how can a given peer use the information on direct and recommenda- 
tion experiences between the peers, that it can retrieve from the network, to 
evaluate the trustworthiness of any other peer? A bit of caution is needed here: 
we do not say that every peer knows the whole trust multigraph. Instead, we 
assume that it is reconstructed (the whole graph or a part of it) by querying the 
other peers, which may misreport. Generally, the direct experiences of the nodes 
which interacted with the given node (nodes u and v in the case of node j from 
Figure 1) should be propagated through the graph down to the computation 
source (node i) by using the recommendation experiences along the paths to 
filter them out. Different works propose different strategies for doing this. We 
classify them in Section 3. We stress that most of the existing works do not model 
explicitly the context of recommendations but rather use direct experiences as 
filters. This can be thought of as weighting one’s reports by his trustworthiness 
rather than his ability to recommend. As well, we emphasize that most of the 
works use binary (zero-one) evaluations of the individual interactions. In the 
example of Figure 1 this would mean that all the numbers are zero or one. 

It is important to note that the formed trust graph does not coincide with 
the underlying P2P network. We see the underlying P2P system as manag- 
ing a distributed database of (data key, data value) pairs. The problem is how 
to distribute this database among the peers so that the basic database opera- 
tions (e.g. search) are efficient and the storage space required at each peer is 
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small in comparison with the size of the database. Two fundamental approaches 
exist to achieve this: (1) unstructured [6], in which the data is distributed ran- 
domly over the peers and broadcasting mechanisms are used for searching and 
(2) structured [7], [8], [9], that build up distributed, scalable data access struc- 
tures to route search requests. Having clarified this, it should be clear that the 
trust graph can be actually stored in the underlying P2P system. In the case 
of an unstructured P2P overlay every peer can store its outgoing edges from 
the trust graph (the identifier of the destination node and possibly time stamp 
may act as the key), while in the case of a structured P2P overlay the triples 
(i destination , source, timestamp ) may act as the keys for the trust graph edges 
and be stored at peers just as dictated by the P2P network [10]. In both cases 
weights of the edges may act as the values. Thus, exploring the trust graph 
reduces actually to searching the underlying P2P network. More specifically, re- 
trieving feedback about any specific peer is subdued to searching for the data 
items with the keys starting with that peer’s identifier. This can be done effi- 
ciently in a structured P2P network. In the rest of the paper we will assume this. 



3 Trust Models Classification 

3.1 Classification Criteria 

A clear categorization of P2P computational models of trust based on managing 
peers’ reputations must consider in the first place the models’ behavior with 
respect to the following three dimensions: the incurred implementation costs, 
the resistance to various attacks and the trust related model semantics. 

As P2P networks normally involve millions of nodes particular attention 
should be paid to cutting down the total implementation overhead introduced 
by the employed reputation management solution. It consists of: the communi- 
cation costs associated with the process of retrieving the necessary feedback, the 
involved storage costs, and the computation overhead related to the feedback 
aggregation. Resistance to attacks normally implies an analysis of the model 
responsiveness to various forms of misbehavior of the peers. The following two 
types of misbehavior have been established in the literature as relevant: indepen- 
dent cheating in interactions or badmouthing other peers and forming collusive 
groups to badmouth specific peers and boost trust values of some other ones. 
For example, peers u and v in Figure 1 may misreport their experiences with 
peer j independently or they may collude and misreport in correlated ways. 

The last mentioned dimension deserves more explanation as it is accompa- 
nied by quite some disagreement in the literature. Deriving from [11], we view 
trust as being inseparable from vulnerability and opportunism associated with 
the interacting parties. Consequently, we say that peer A (trustor) trusts peer 
B (trustee) if the interaction generates a gain to be shared with and by peer B 
and exposes peer A to a risk of loss, if peer B takes a too large portion from the 
joint gain. Building on this, we see trust management as a set of actions related 
to: 1) reducing the opportunism of the trustee, 2) reducing vulnerability of the 
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trustor and, after these two issues have been properly addressed, 3) deciding if 
and when to enter an interaction. 

Clearly, the main goal of any reputation management mechanism is partial or, 
if possible, complete reducing of the opportunism of the interacting parties. The 
degrees at which different mechanisms achieve this vary. We see the following 
classes: social networks formation, probabilistic estimation techniques and game- 
theoretic models, which are discussed next. 

3.2 Social Networks 

This class of approaches normally implies that the entire trust multigraph is 
reconstructed and that all reputation information available in it is aggregated 
(Figure 1). A natural interpretation of the aggregation process involves the fol- 
lowing steps: 1) enumerating all paths from the trust computation source to the 
target node, 2) aggregating the trust values along the paths to give a path wide 
gossip and 3) merging these gossips into a final value. 

Where is the exact position of this class with respect to the tree dimensions 
introduced above? 

Normally, their implementation overhead is high. Because a trust-computing 
node has to retrieve the entire trust network, the communication costs are very 
high. Besides, because the number of paths between the trust-computing source 
and target can be exponential, the reputation aggregation process is too costly. 
These two problems are even more strongly emphasized if the context of recom- 
mendation is present - see [12] for an example. 

[13], modeling only direct experiences, offers important theoretical insights on 
this issue by characterizing the combinations of path and across-patlr aggrega- 
tion strategies that may lead to a non-exponential trust computation algorithm 
(we note that many other works use such combinations: e.g. [14], used for Web 
pages ranking, and [15]). Central to the approach is the claim that, for specific 
combinations of the aggregation strategies, exploring all the paths is equiva- 
lent to finding a convergent power of the trust matrix, derived naturally from 
the trust graph. 1 (Here, the matrix “multiplication”’ operation is derived from 
the path and across-path aggregation operations.) However, the proposed al- 
gorithm requires the synchronous participation of all peers, making it hardly 
implementable in a P2P network. Instead, we believe that an incremental com- 
putation is something worth further investigation. 

[16] offers important insights with respect to this. The gist of this approach 
consists of computing the trustworthiness of a given node as the average of 
its performances as seen by its neighbors in the trust graph, weighted by the 
trustworthiness of the neighbors themselves. The authors also develop a simple 
caching scheme in which the trust values of the neighbors of the trust computa- 
tion target are taken from a cache (default values are used in the case of cache 



1 It is assumed that the interaction multigraph is transformed into a graph by aggre- 
gating first the interaction outcomes between the pairs of nodes. 
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miss) and their computed trust values replace the corresponding values existing 
in the cache. 

How robust is this class of approaches in presence of various misbehaviors? 
[15] and [16], the only works providing informative simulation results, report 
good performance of the corresponding approaches when the fraction of mali- 
cious peers is small (below 45% approximately) and the malicious peers inde- 
pendently cheat in the interactions and distort their ratings of other peers. [16] 
further reports the complete breakdown of the mechanism when the cheaters 
take more than a half of the overall population or when they collude. On the 
contrary, [15] claims almost full effectiveness of their mechanism when the ma- 
licious peers make the larger fraction of the population and collude in various 
ways. This results from, in our opinion, the fairly unrealistic assumption that a 
number of pretrusted peers exist each of whom is assigned some non-zero trust 
by the rest of the community, including the malicious peers. (This is so called 
“random walker” model used for Web pages ranking.) 

The computed values have unclear semantics and are hard to interpret. They 
cannot be interpreted as the (estimated) probabilities of the trustworthy behav- 
ior of the target peers and the question what exactly they represent is left open. 
Let us also mention an interesting detail related to [13] and [15]: when the trust 
graph is irreducible and apperiodic the powers of the corresponding trust matrix 
converge to a matrix in which all the rows are the same and sum up to 1 (the 
primary eigenvector of the matrix) . Thus the trust values of the peers have global 
meaning - they are independent of the computation source. On the other hand, 
because all the values sum up to 1, it seems as if the trust was distributed among 
the peers. But, if we have the values for all the peers and they are approximately 
close we are in doubt whether the whole network is trustworthy or it is malicious. 

This leads us to conclude that the computed values lack a plausible inter- 
pretation on an absolute scale and that the only scenarios in which they can be 
used must involve ranking the trust values of many peers and selection of the 
most trustworthy one(s) among them. 

3.3 Probabilistic Estimation 

Probabilistic estimation techniques present certain improvement with respect to 
the meaningfulness of the computed values. Namely, they output probability dis- 
tributions (or at least the most likely outcome) over the set of possible behaviors 
of the trusted agents. The importance of such models becomes clear if we recall 
the presented view on trust - if the opportunism of the trustee cannot be reduced 
completely then it becomes important for the trustor to be able to estimate the 
risks of the interaction and decides whether to enter it or not. If the individual 
outcomes of the interaction are assigned the probabilities and the trustor can as- 
sign them utilities as well then this task becomes easy - the trustor just needs to 
compute whether entering the interaction has a higher utility than staying out. 

In principle, it is possible to construct a probabilistic model in which all the 
paths between the trust computation source and target. However, we are not 
aware of any such attempt and doubt that it would suffer from an exponential 
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computation overhead, just as outlined previously. But, we believe that con- 
structing such a model is unnecessary and that in most of the relevant settings it 
is sufficient to consider only two small fractions of the formed trust (multi) graph 
- those around the trust computation source and target. One of the goals of this 
paper is exactly to show this. Needless to say, another clear advantage of doing 
the trust computation this way is its implementation efficiency. 

It is a bit surprising that very few works on using well known probabilistic 
estimation techniques for decentralized trust computation exist. [17] presents 
the well-known method of Bayesian estimation as the right probabilistic tool for 
assessing the future trusting performance based on past interactions. Only direct 
interactions were studied - the question of including recommendations was not 
considered. 

[18] goes a step further by taking into account the “second-hand” opinions 
also. However, the strategy for merging own experiences with those of other 
witnesses is intuitive (giving more weight to own experiences, though plausible, 
is still intuitive) rather than theoretically founded. 



3.4 Game-Theoretic Models 

Game-theoretic reputation models make a further clarification in the interpre- 
tation of the agents’ trustworthiness in the sense that, if the reputation system 
is designed properly, trust is encoded in the equilibria of the repeated game the 
agents are playing. Thus, for rational players trustworthy behavior is enforced. 
The real challenge here is how to define the feedback aggregation strategies that 
will lead to socially desirable outcomes carrying trust. 

[19] presents the proper game-theoretic framework for analyzing reputations 
(repeated games with incomplete information), while [20] and [21] offer certain 
characterizations of the equilibria payoffs in the presence of reputation effects. 

[22] focuses on a specific game and derives its equilibria. Apart from this the 
author also raises questions concerning the overall game-theoretic reputation 
systems design, such as incentivizing players to leave feedback, dealing with 
incomplete feedback etc. However, an underlying assumption of this work is that 
a central trusted authority does the feedback aggregation. We see this as a major 
obstacle to transferring game-theoretic models to decentralized environments. 



4 Model I - Honest or Dishonest Peers 

Let us now consider a P2P network consisting of peers having associated innate 
probabilities of performing honestly in their interactions with others. Let 9j de- 
note the probability of peer j. Assume that peer j interacted with peers pi,...,p n 
and its performances in these interactions were X\, . . . ,x n , where Xi £ {0,1} 
(1 denoting the honest performance and 0 the dishonest one). When asked to 
report on peer j’ performances witnesses pi,P 2 , ■ ■ ■ ,Pn may He and misreport. 
Assuming that they lie with specific probabilities, say Ik for peer pk, the proba- 
bility of observing report yk from peer pk can be calculated as: 
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P[Y k = Vk] 



lk( 1 - Qj) + (1 - h)0j if yk = 1 
hOj + (1 - h)( 1 - Oj) if y k = 0. 



(1) 



Now, given a random sample of independent reports y i , 2 / 2 » - - - ,Un we have 
that the likelihood function of this sample is 



L{9j) = P\Yi = yi]P[Y 2 = y 2 ] • • • P[Y n = y n ] . (2) 

The maximum likelihood estimation procedure now implies simply finding 6j 
that maximizes this expression. This number is the maximum likelihood estimate 
of the unknown probability. Note also that the own experiences are seamlessly 
integrated into this model - the trust computing source peer i just has to put 
Pi = 1 for his own experiences ay. 

Let us locate this model in the three-dimensional space introduced in Section 
3.1. 

First, referring back to Figure 1, if peer i is computing the trustworthiness of 
peer j then the model assumes the peer i first retrieves from peers u and v their 
reports on peer j's performances with them. (This is the meaning of y±, y 2 , . . . , y n 
in (2)). Thus, the necessary reputation information on which the model operates 
consists in this case of the edges entering node j. Keeping in mind what we said 
in Section 2 about how the overlay network is used to store the trust data we see 
that retrieving these edges (feedback) coincides with searching the overlay for 
the data items with the keys starting with j. Further, as we will see shortly, not 
all such data items have to be retrieved. Good predictions can be achieved even 
with 10-20 reports retrieved. We stress that this is a considerable improvement 
as compared to what most of the existing approaches do - we retrieve only a 
small fraction of the feedback about the trust computation target, while they 
retrieve the entire feedback about all the nodes. 

On the other hand, we assume that peer i deduces the misreporting proba- 
bilities l k from own interactions (by comparing its own performances with the 
reports about them). These “averages” can be maintained for specific peers (l k s 
are different) or at the level of the whole network (l k s are all same). In the simu- 
lations below we assume the second possibility. In this case the involved storage 
costs per peer are negligible - only a single value is kept and used to approximate 
the situation in the network. 

Thus, in a word, the model incurs small communication overhead and virtu- 
ally no storage costs. 

Second, the output values of the model are probabilities and as such they do 
have a plausible interpretation on the absolute [0, 1] scale. Therefore, it is easy 
to interpret and use them without comparing with the other peers’ values. 

Third, Equation (2) implies the independence of the reports Y 1 ,Y 2 , . . . ,Y n . 
Thus, the assumed setting is non-collusive and the simulation results we present 
next hold for this setting. We note that extensions to collusive settings are pos- 
sible by integrating the collusion possibilities into (2). 




70 



Z. Despotovic and K. Aberer 



4.1 Simulation Results 

We checked the performance of the method in a variety of parameter settings. 
As the estimation quality measure we chose the mean absolute error of the 
estimated probabilities of the peers performing honestly and their actual val- 
ues. The following parameters are considered: 1) the number of peers - 128 
(constant throughout the simulations), 2) the number interactions per peer 
- varied at increments of 20 from 20 to 100, and 3) fraction of liars - var- 
ied at increments of 0.1 from 0.1 to 0.5. All the results are averaged across 
20 simulation runs. The interactions among the peers were generated at ran- 
dom, we did not consider any particular structure of the resulting trust net- 
work. 

Note that the second parameter, number of interactions per peer, is not corre- 
lated with the numbers of peers. For higher network sizes the same results would 
be obtained with the same numbers of interactions per peer. Put differently, the 
absolute amount of feedback is what determines the results, not its relative size 
as compared to the size of the network. 

Figure 2 shows the results for the case when the peers’ probabilities of per- 
forming honestly are generated at random in the interval [0, lJ.Note that the liars 
were generated so that they always lie (Ik from (1) equals the fraction of liars). 
We have also experimented with varying IjS and did not observe any important 
difference. 



Mean absolute 




Fig. 2. Simulation results - peers’ trustworthiness drawn randomly from [0,1] 

We emphasize that the plot is symmetric across the line “fraction of liars = 
0.5”. This is simply a consequence of the introduced probabilistic assumptions 
and non-collusive peer behavior - if a peer believes that the majority of the peers 
are liars then it should take the reverse of their reports as true. Interestingly, no 
existing approach exhibits this behavior. 
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5 Model II - Normally Distributed Services 

There are settings in which the peers’ behavior or their performances within 
certain contexts are characterized by a globally known finite number of types. An 
example of such a setting would be the speeds of the network connections of the 
peers. Namely, there are a fixed number of the connection types with prespecified 
speeds. (However, the speed of any given connection is only theoretically fixed, 
in practice it varies due to different conditions in the network.) In this section we 
present a general model of such settings and show how the maximum likelihood 
estimation can be used in this case. The simulation results, given in Section 5.2, 
show that the technique performs even better than in the setting of Section 4. 
Intuitively, this is a consequence of the stronger constraints imposed on the peers 
behavior. 

Consider a P2P network consisting of N peers Pi, ■ ■ ■ ,Pn- They provide ser- 
vices to one another and we are assuming that each observed service quality 
is distributed according to a normal distribution A 1 < i < N s . Thus, 
there are N s different distribution from which the service qualities are drawn, all 
of them having the same standard deviation a but different means p \, . . . , p,N s . 
These distributions will be denoted by A/) so that, for any 1 < k < N s the 
symbol A 4 will denote the normal distribution A f(pk,cr). 

We further assume that the means are innate to the peers so that whenever 
a given peer, say peer pj , provides a service then the service quality observed by 
its consumer will be a random variable distributed according to M{pk , c) , where 
/Life is peer Pj’s mean. 

As in the precedent model, we are assuming that a fraction of the peer popu- 
lation pi are liars, while the rest of the populations are honest participants. The 
fraction of liars is learned by comparing the own performances and the reports 
about them. We will assume the following behavioral patterns of the two groups 
of peers. Consider an interaction between service provider pi and consumer pj 
in which pj observes the quality level x. If peer pj is honest then it will report 
value x when asked to report on peer pi s. On the other hand, if pj is a liar then 
it will first choose a distribution A/fc at random and then report a random value 
generated from this distribution. 

Our task is, just as before, to show that a maximum likelihood based esti- 
mation method can be used to accurately predict the future performance of the 
service providers given the reports of their past provided qualities. Note that a 
very similar setting was considered in [23], where a machine learning technique 
was used instead. 

5.1 Estimation Derivation 

Let 6 denote the unknown mean of a given peer. To determine the likelihood of 
any report on that peer’s performance we must first determine the distribution 
of the reports. Denoting by Y the corresponding random variable we have: 

P[Y <y}= piP[Y < y\ false report] + (1 — pi)P\Y < y\ correct report], (3) 
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where 



and 



N, 



P[Y < y | correct report] = ^^P[Y < y\Y is Af(9,cr)\ 



i= 1 



N a 



P[Y < y | false report] = ^ — — P[Y < y\Y is TV];]. 

. -i » c 



( 4 ) 



( 5 ) 



Taking derivatives we obtain the following probability density function: 

^ 1 1 (»-ni) 2 , N 1 >y-o> 2 

fy(y)=Pi2^ 1 r T 7=e 2.2 +(1 -pi)—^= e 2.2 . (6) 

2 V S ay Z7T cry 27T 



Now, having a random sample j/i, J/ 2 , • ■ • , 2/n from this distribution and assum- 
ing that the independence of the samples we compute the likelihood function: 



L{0) = /y(z/i)jy ( 2 / 2 ) • • • friVn)- (?) 

The final task is to select the unknown parameter 9 £ {yi, y -21 ■ ■ ■ , such 
that this likelihood function is maximized. 

We stress that whatever has been said in Section 4 on the the model’s main 
properties applies also here. In short, low implementation overhead and mean- 
ingful interpretation of the output values remain the main properties of this 
model too. Its prediction quality is given next. 



5.2 Simulation Results 

We now present the results from a series of tests that show the performance 
of the method. We first generate peers, determine the liars and assign service 
distributions to the peers. The services are generated in such a way that, if there 
are N s services then their means are random numbers between 1 and N s (the 
standard deviation was varied between 0 and 1 as presented below). Being a liar 
and having any specific service are not correlated. Then we generate a number 
of interactions at random. The following parameters are considered in the sim- 
ulations: 1) the number of peers - 128 (constant throughout the simulations), 

2) the number interactions per peer - varied at increments of 20 from 10 to 50, 

3) fraction of liars - varied at increments of 0.1 from 0.1 to 1.0, 4) number of 
services - 4 (constant throughout the simulations) and 5) standard deviations of 
the service distributions - varied at increments of 0.1 from 0.1 to 1.0 with the 
default value of 0.3. The fraction of incorrect guesses (misclassification rate) is 
used as the quality measure. 

Figure 3 presents the dependency of the misclassification rate on the fraction 
of liars and the number of interactions per peer considered. The number of 
services was 4 and their means were all 0.3. 

Figure 4 shows how the misclassification rate depends on the standard devi- 
ations of the services. 
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Fig. 3. Misclassification rate of the expected service quality as a function of the fraction 
of liars. Standard deviations of the service distributions - a = 0.3 
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Fig. 4. Misclassification rate of the expected service quality as a function of the stan- 
dard deviation of the distributions of the services 



Let us comment on the presented results. First, because the number of the 
services is greater than two and the liars lie uncoordinatedly good predictions 
can be made even when the fraction of the liars is close to one. Second, the 
method becomes very robust as the number of interactions considered increases. 
When the inluence of the “noise” (standard deviation a) is not too high, the 
method is able to separate the distributions quite well. 

Though the simulation setting considered in this section is different from the 
setting of Section 4 we can still derive some conclusions by comparing the pre- 
sented results. The main difference between the two models is as follows: the 
model in the previous section assumes infinitely many behavior types (probabil- 
ity distributions) , while the model in this section implies a known finite number 
of behavior types. It is this difference that enables a better performance of the 
maximum likelihood estimation in the second case. 
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6 Discussion and Future Work 

We propose in this paper a simple probabilistic method to assess peers’ perfor- 
mance in a P2P network (or more specifically their trustworthiness). The method 
is based on the well known statistical estimation technique - maximum likelihood 
estimation. We test its quality on two settings relevant for P2P communities. 
From the test we derive the following as the main properties of the method. 

1. The method is able to estimate peers’ characteristics (i.e. their trustwor- 
thiness) quite well. For smaller fractions of liars, the method gives good 
estimates even when a small number of recent interactions is considered 
(around 20). This is particularly important when the assumed peer behavior 
is time dependent. In this setting, our method is more capable of capturing 
the behavioral dynamics. 

2. The implementation overhead of the method is as small as possible. Practi- 
cally, it only implies the small communication overhead related to retrieving 
the direct experiences of the peers who interacted with the trust computation 
target. 

3. The considered setting was non-collusive; it assumed that peers did not form 
collusive groups but rather acted independently. In this setting, the mecha- 
nism gives estimates of the peers’ trustworthiness with errors within 5-10% 
even with 30% of liars. 

To the best of our knowledge, no existing approach exhibits such properties. 
At best, they show similar quality of the trustworthiness estimation (similar or 
higher estimation error) but require substantially higher implementation over- 
head. On the other hand, most of them can be applied only in specific settings 
(e.g. file sharing) because they essentially need ranking of the trustworthiness 
values of various peers. Our comparison of the results given in [15], [16] and 
[23] and those presented here confirms these claims directly. There is a simple 
explanation of this. Namely, the simulation settings we consider coincide (or are 
very similar) with those presented in the mentioned works. This is true not only 
for the simulation parameters but also for an important aspect of the assumed 
behavior of the peers. Precisely, all the works assume that the peers are asso- 
ciated with specific probability distributions that determine their performance. 
The key difference between our and the other solutions is that we introduce this 
assumption explicitly into the trust computation method, while the other works 
do not. 

However, we did not deal in this paper with the case in which these proba- 
bility distributions are correlated among the peers. Put differently, we assumed 
here independent acting of the peers, without forming collusive groups. Check- 
ing precisely the performance of the presented method in collusive settings and 
extending it to be as effective as possible under this assumption on the peers’ 
behavior make the most important part of the future work. This can be done 
either by modeling collusions probabilistically (operating directly on Equations 
(2) and (7)) or learning the probabilities of misreporting for every peer in the 
network separately rather than for the network as a whole. 
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Abstract. In this paper we evaluate the performance of MAIS, a multi-agent 
system for searching the Internet. The search for interesting documents on the 
Internet is becoming more and more difficult. Inference engines are largely 
outperformed by the expansion of the Internet and quite often the user is not 
satisfied with the results of a search. We believe that multi-agent systems can 
help to improve the user’s satisfaction because in a multi-agent system (MAS) 
agents can be specialized in different tasks (search or filtering) and can person- 
alize information. We implemented such an approach using an open MAS con- 
taining personal assistants, library agents, filter agents and search agents. In this 
paper we study the elements of our MAS and compare the performance of the 
system with traditional search engines taking the viewpoint of the user. Our re- 
sults are much more accurate than standard search engines. 

Keywords: Personalized Information Retrieval, Information Agents, Web 
Search 



1 Introduction 

Savoy [22] indicates that 85% of the users searching the web for information use several 
search engines, which is a clear indication of the limits of the search engines on the 
Internet. The differing results for the engines are due to two main factors: (i) search 
engines have different strategies to evaluate the quality of the sites in relation to a user’ s 
queries, and (ii) search engines databases contain different sets of indexed pages. 

The quality of a site is estimated by a relevancy measure that ranges from simple 
term frequency-based techniques to more elaborate heuristic strategies. A good 
example is Google. Google considers that a web site is important when it is often 
referenced by other web pages — PageRank technique. Consequently, web addresses 
present in many pages have priority over less known addresses [4]. We think that if 
such generic strategies may work for some users, they cannot work for most of them, 
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essentially because they do not take into account the user’s profile or background in 
order to obtain better, personalized results. 

The personalization on the web is not a new research topic. Web personalization 
was first undertaken in the nineties by Lieberman [13], Armstrong et al. [1] or Chen 
and Sycara [5] who introduced agents for searching the web, while recording the 
user’s moves. In such approaches, the visited web pages were treated as a database 
containing common topics. Existing links helped selecting new pages automatically. 
Such applications are not actual multi- agent applications since distributed exchange 
of information is not good enough to qualify a multi-agent system. Furthermore, they 
can be developed using traditional distributed objects or client/server techniques. On 
the other hand, in a multi-agent system, agents share an interaction protocol (not only 
a communication protocol) to accomplish common tasks like building coalitions, 
allocating tasks, or doing coordination. 

In this work we compare the performance of a multi-agent system developed for 
searching the web, MAIS (Multi- Agent based Internet Search) [6]. In MAIS, agents 
with different roles are used to distribute tasks in an open environment. Personaliza- 
tion in MAIS is accomplished analyzing favorite sites and using public interest cen- 
ters. The paper is organized as follows: Section 2 introduces agents for the Internet; 
we discuss some strategies for personalization in Section 3; Section 4 presents some 
details of the MAIS system; in Section 5, we evaluate the performance of MAIS 
comparing results with traditional search engines; in Section 6 we discuss related 
works and, finally. Section 7 concludes the paper. 

2 Multi-agents and Internet 

This section first addresses the advantages one could benefit from developing multi- 
agent systems for Internet applications. 

Using multi-agent systems may have a number of advantages like scalability, sta- 
bility, or load balancing. 

• Scalability: the system can be updated or expanded without affecting the normal 
operations. Since agents are independent systems, they can be created and modi- 
fied individually while other agents continue to provide services 1 ; 

• Stability: whenever an agent crashes, other agents can coordinate the distribution 
of services and ensure their execution; 

• Load balancing: agents distribute processing naturally. Some tasks like learning 
profiles or searching the web can be very hard indeed. Agent Coordination Pro- 
tocols tend to distribute the work between different agents running in different 
processes, threads, machines or networks uniformly. 

Nowadays, much attention is paid to information or gathering agents [15]. Such 
agents can follow the context of the user and provide personalized information to 
users using a device connected to the web (telephones, notebooks, palms, PDAs, etc.) 



1 Note that this does not say anything about what happens to the bandwidth when the number of 
agents increases drastically. 
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in professional or entertainment applications [11] [19]. Other less frequent applica- 
tions are based on distributed problem solving. Such agents can compete for re- 
sources (booking hotels, flights, restaurants, or cinemas) or collaborate to satisfy the 
user (e.g., Freitas and Bittencourt [9] give an example for searching specific informa- 
tion like call-for-papers on the web with a collaborative multi-agent system). 

Despite the enormous potential, we do not see many multi-agent systems for pro- 
viding Internet services, although this situation is changing with emerging multi- 
agent standards like those proposed by FIPA [8]. There is however a general consen- 
sus that intelligent agents will guide users in many activities related to the web [15]. 
Many projects are being undertaken with this idea. For example, the Agentcities Task 
Force [25] congregates more than fifty FIPA compliant agent platforms spread 
throughout the world for providing services. Currently they offer only basic services 
for the purpose of evaluating standardization. In the future, such platforms could 
provide intelligent services. Another example in the multi-agent community is the 
Trade Agent Competition — TAC [25]. TAC is a public competition between several 
trading agents in the world. In each competition, eight agents compete against one 
another. Each agent must use trade strategies to maximize the resources of its users. 
Resources allow organizing travel and must be sufficient for buying airline tickets, 
making hotel reservations and optionally spending for entertainment. TAC is a good 
illustration of the services that could be provided by Internet agents in the future. 

We exploit the advantages discussed in this section with a system for personalized 
search on the web. The agents share information about the users, coordinate the task 
allocation (filtering and search) and distribute the processing, enabling the treatment 
of several queries in parallel by different machines. 

3 Adaptive Agents and the Web 

Agents are often used in the implementation of “intelligent” Internet services. Such 
services can aid the user to personalize news, find papers, search sites or filter infor- 
mation. Since the nineties, many applications has been developed with such features. 
Most of those applications have in common the usage of an accurate user model for 
improving and personalizing results. Middleton [17] discusses more than sixty appli- 
cations, organizing them according the different implementations of the user model, 
namely: 

• Supervising the User Behavior: the agents represent users’ actions on unsuper- 
vised databases then, an incremental learning method (for instance, behavior 
networks or memory-based algorithms) is used for identifying the sequences of 
actions adapted to a given situation; 

• Using Explicit Feedback: the agent ask the user for information for correcting 
errors and refining the answers to the problems; Lieberman [14] calls this 
approach “Profiling by Example;” 

• Learning from a Base of Experiences: the agent constructs a database of ex- 
periences and a non-incremental learning algorithm produces a user model; 
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• Allowing the User to Program the Agent: the user can program the agent for 
improving its performance; 

• Using a Knowledge Base: the agent can use predefined user models. The user 
can fill forms and give answers to general questions. Based on this information 
the user constructs a knowledge base. 

Next section presents a web search application that uses the first two techniques for 
modeling the user. The user model is updated dynamically, based on the user’ s actions 
and feedback. For the web search, the user’s actions are limited to adding and removing 
sites to and from favorites and changing the centers of interest. Additionally, the user 
can give feedback, adding sites presented in the result of the searches as favorites. 




4 MAIS Architecture 

In a previous paper [6] we presented a multi-agent system for searching the web, 
MAIS (Multi-Agent system for Internet Search). In this section, we discuss some 
details of MAIS. The architecture of MAIS is composed of several agents: Personal 
Agent (PA), Library Agent (LA), Filter Agent (FA) and Search Agents (SA). These 
agents are presented Fig. 1. 

The user interacts with the system using a PA interface (Fig. 2). Currently, the inter- 
face of the system is a prototype and we are not concerned with the visual aspects. The 
user can also select some public centers of interest using this interface (Fig. 3). Each 
interest center is composed of a fixed collection of documents selected by experts and 
of the dictionary learned with LA. For instance, Artificial-Intelligence-Adaptive-Agents, 
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Fig. 3. Public centers of interest 

Groupware-Knowledge-Management and Artificial-Intelligence-MBR are examples 
of interest centers present in the system. Such interest centers are shared by all users, 
but can be personalized progressively by adding new documents to the profile. The 
documents in an interest center are copied into the user space when the topic is se- 
lected. Since the processing of requests and the user data are distributed among sev- 
eral agents, the copies do not affect the system scalability significantly. Then the user 
can update the interest center. Currently, the number of available interest centers is 
small but we expect to build a collection of interest centers similar to the directories 
organized in categories commonly available in web search engines. 
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The PA provides two services: the management of the favorite sites and the man- 
agement of the search. 

4.1 Management of Favorite Sites 

Favorite sites are stored in the user workspace inside folders as text documents. The 
Library Agent (LA) analyzes the documents and constructs a general dictionary to 
represent them. The resulting structure contains the most important terms extracted 
from the collection of personal documents. Similarly, another LA is used to construct 
the dictionary of the public interest centers. 

4.2 Search Process 

The user starts a search passing information to the PA, i.e., a list of terms, the maxi- 
mal number of sites in the answer and the interest centers. Then, the PA sends a 
broadcast message and contracts a Filter Agent (FA) to start the search. 

The FA plays a central role in the system. An FA engaged in a contract must re- 
trieve sites from Search Agents (SAs). The FA sends a call-for-bids, passing the 
user’s terms, the number of sites and a deadline to all SAs. The FA waits for all an- 
swers to arrive before a time limit. In the next step, it starts the analysis of the sites 
for personalizing the final result. 

In order to personalize results, the FA orders the list of sites received from SAs 
according to the user’s preferences. Each page is submitted to a classification process 
that assigns a degree of interest to the page. The page classes are the favorite folder 
names and the interest centers of the user. The sites are then ranked and the best ones are 
selected. Next sections give more details on how the Library and the Filter agents have 
been implemented. Search Agents retrieve sites from search engines quite simply. 

4.3 The Library Agent (LA) 

The LA is specialized in document management. It constructs a general structure (a 
dictionary) from a collection of document. The LA uses automatic document classifi- 
cation or information retrieval techniques. Documents are represented as vectors of 
important terms, using TF-IDF (term frequency - inverse document frequency) 
measure [21], 

4.4 The Filter Agent (FA) 

The FA coordinates the exchanges among the agents of the system. It is responsible 
for personalizing the list of sites received from the SAs based on the user profile. The 
FA uses ELA (Entropy-based Learning Algorithm) for classifying the sites. ELA is a 
lazy graph based learning algorithm. Details about ELA can be found in [7]. 

The classification of a site returns the class of the site and a classification rate. The 
classification rate is the value computed in the ELA algorithm. Before using the 
learning algorithm we need (i) to compute a general structure for the user model and 
(ii) to construct a training set. 
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Fig. 4. Combination of the document structures in a global document representation 

4.4.1 General User Model 

The structure of the user model takes into account the representation of the user fa- 
vorite documents and the representation of each center of interest. We merge such 
structures, ordering the terms according to the TF-IDF values of the terms and limit- 
ing the dimension of the vector arbitrarily to 1000. With this approach we guarantee 
that terms, important for all the classes, are present in the general structure. Fig. 4 
gives an example of the process. 

Thus, the general user model representation consists of a list of terms selected 
from ( n + 1) vectors representing document collections: 1 for the user favorites and n 
for the number of interest centers selected previously. In this process, the system does 
not analyze the content of the documents, but simply uses the vector model of each 
document collection, saving much processing time. 

4.4.2 The Training Set 

We use the favorite document vectors and the interest center document vectors to 
construct the training set for the ELA algorithm. We stress that the class for each 
vector is known (the name of the favorite folder or the name of the interest center). 
Since the current version of ELA (Entropy-based Learning Algorithm) works strictly 
with symbolic data, vectors are formed of binary variables. 

4.5 MAIS Construction 

OMAS 2 agents [2] have been used in the construction of the system. OMAS agents 
live in an open environment where agents communicate using KQML messages 
transmitted with a UDP protocol. The agents always communicate in broadcast mode. 
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2 Open Multi-Agent System architecture. 
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The content language is generally coded in LISP structures and the interaction proto- 
col is Contract Net [23]. 

4.6 MAIS Interoperability 

A Transfer Agent (TA) guarantees interoperability between OMAS platforms and others 
FIPA compliant platforms in a transparent way. The TA implements some basic FIPA 
services like Agent Manager System, Directory Facilitator and Agent Communication 
Channel. The TA can translate OMAS messages in SL-0 messages and make OMAS 
and FIPA protocols compatible. Consequently, the TA is the bridge between OMAS 
agents and remotes platforms. Fig. 5 presents the architecture of the system. 



OMAS Platform 




FA - Filter Agent LA - Library Agent 

SA - Search Agent TA - Transfer Agent 

Fig. 5. Communication between the OMAS platform and FIPA platforms 

From a technical point of view, the Personal Assistant (PA) is composed of a LISP 
interface, capable of sending messages to the TA that works on a remote HTTP server. 
Then, the TA sends the messages to the internal platform. We use Allegro Common 
Lisp™ as the HTTP server in this application. The application has been entirely coded 
in LISP. Because the web server is a LISP program, the interpretation of KQML and 
SL-0 structures is extremely easy. Fig. 6 presents the resources used in the application. 

To make the system interoperable, we have developed a small language to communi- 
cate between the LISP interface and the HTTP server. Similarly, this language allows to 
communicate with agent platforms other than OMAS [2], used in this work. We tested 
interoperability of our OMAS agents with JADE [12], another FIPA [8] compliant plat- 
form. 

Fig. 7 shows how searches can be started from JADE agents. In the left hand-side of 
the window we have information about our platform. The right hand-side the window is 
used to send a request for sites. The “searcher@OMAS” agent plays the role of the 
assistant and starts the search process. “START-REQUEST” is a term from the content 
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language we developed. The complete language specification it is out of the scope of 
this paper. 




Fig. 6. Software resources of the system 
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Fig. 7. Executing searches remotely with JADE agents 



5 MAIS Evaluation 

Evaluation of web search engines is always a difficult task. Common measures used 
in systems based on information retrieval, like recall and precision, are not adapted 
due the dynamic behavior of the web corpus. Thereby, most works use a limited 
corpus of web pages. A common strategy is to score web pages using a similarity 
measure between results and training pages. For instance, Menczer et al. [16] used refer- 
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ence web pages organized in 100 categories proposed by Yahoo for the comparison 
of three search engines. Nevertheless, the page datasets used for learning and test are 
small and static and is not a good sample of the Internet data. To avoid such prob- 
lems, Gasparetti and Micarelli [10] used recall and precision measures on the “.GOV” 



Table 1. Information about the users 



User 


Expertise 


Number of reference 
web pages 


Interest Centers 


A 


Multi-Agent Systems 


55 


Adaptative- Agents 
Autonomous-Agents 
Agents 

Personal Assistant 


B 


Artificial Intelligence 


8 


Adaptative- Agents 
Autonomous-Agents 
Agents 

Personal Assistant 


C 


None 


1 


- 


C-int 


None 


1 


Adaptative- Agents 
Autonomous- Agents 
Agents 

Personal Assistant 



database provided by TREC 3 (Text REtrieval Conference). This database has got 1.2 
million of pages “.gov” collected until 2002. According the authors, the database is 
large enough to represent a feasible sample of web data. The authors evaluated a web 
page classification algorithm using three queries: “exchange office,” “educational 
department” and “children foundation.” The average number of results for each query 
is 373712, 445964 and 117036 respectively. We claim that in a realistic approach, an 
answer must contain a small number of interesting web pages, because very often the 
user will analyze only few addresses appearing at the top of the list [24]. 

To avoid problems related to recall and precision measures, in our evaluation 
approach, we use the feedback of three users for evaluating the results obtained directly 
from the web. Thus, we do not use simulated databases. The users analyze only the top 
twenty or thirty web pages telling how many pages are interesting to them. All users 
search for information about multi-agent systems and a site is considered interesting if it 
is related to multi-agent research domain. Each user selected a number of reference web 
pages before the experiments. Table 1 presents more details about the users. 

Each user sent the following queries: “agent,” “autonomy agent,” “personal assis- 
tant” and “personal assistant agent.” The queries are intentionally ambiguous. The 
goal of the evaluation method is to establish the importance of the users’ expertise 
used as reference for the information filtering process. We also measured the influ- 
ence of public interest centers proposed by the system through user C. User C was 
invited to send the queries with no interest centers support and, further, using the 
interest centers (C-int). 



3 http://trec.nist.gov 
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The graph presented Fig. 8 presents the performance of three widely used search 
engines (All-the-Web, Altavista and Google) and the results of the users A (A-55) 
and B (B-8) that used MAIS. As expected, user A obtained the best results, showing 
that the system is capable of taking into account the user context represented as favor- 
ite sites for personalizing results. User B similarly also benefited from this advantage. 



Performance with 20 sites 
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Fig. 8. Performance with the 20 first sites 



Two observations seem clear based on the results of Fig. 8: 

• The first one is the limitation regarding the performance of MAIS in relation to 
the quality of the search engines; 

• The second one is the variability of search engines. 

Because MAIS does meta-searches, if all search engines used in MAIS do not give 
good results, so does MAIS. This can be observed with the query “agent,” which is 
quite ambiguous. Some search engines present good performances for specific que- 
ries but bad performance for different queries. We observed for instance that All-the- 
Web produces good results for the query “Autonomy Agent” and poor results for the 
query “Personal Assistant.” 

When the first thirty web pages have been analyzed, MAIS gave still the best re- 
sults for user A (Fig. 9). However, other search engines have outperformed the per- 
formance regarding user B in three of four queries. The system cannot support the 
user when the user profile is not compatible with the information he is looking for. 
This can be solved if the user informs the system not to use her profile for the search 
and but only the public interest centers. 

In order to measure the importance of the public interest centers for the informa- 
tion search, user C (whose profile is almost empty) initially sends queries without 
using the interest centers (MAIS(C)) and then queries using the interest centers 
(MAIS-int). Results are shown Fig. 10. For the query “Agent,” interest centers are 
useless because the query is actually quite ambiguous. For the query “Autonomy 
Agent” the interest centers improved the performance significantly when both 20 and 
30 first sites are used. For the last two queries, the performance of the system was 
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improved when the 30 first sites are used. When only 20 sites are used, interest centers 
had not a high importance for the queries "Personal Assistant” and “Personal Assis- 
tant Agent.” This behavior demonstrates that even for reasonable accurate queries 
like “Personal Assistant Agent,” search engines used in MAIS are not capable of 
clustering the best pages and of presenting them at the top of the list. 



Performance with 30 sites 
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Fig. 9. Performance with the 30 first sites 



Performance with 20 and 30 sites, with and without interest 
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Fig. 10. Results with and without interest centers 

Average performance is presented Table 2. Analyzing the 20 and 30 first sites, 
MAIS outperforms all search engines. It shows that the system can benefit from the 
advantages of all specific search engines, producing a more accurate result. Surpris- 
ingly, the winner (C-int) contains an almost empty profile (only one personal page) 
but uses public interest centers. Such results indicate that the sites judged interesting 
by the users do not add important information to the system. In fact, more web pages 
can even decrease the accuracy of the system because they do not contain a lot of text 
and introduce some noise. 
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In addition, the system was more accurate when users selected the public interest 
centers, probably because they are composed of documents selected by experts, the 
documents are representative of the field and contain much textual information. Thus, 
we can conclude that the users will have to carefully analyze the sites he thinks 
appropriate before adding them to his profile. 



Table 2. Average results regarding the number of sites in the results and the number of interest- 
ing sites in the results for the four queries 



With 20 sites 


MAIS(C-int) 


13.25 


MAIS(A-55) 


12.50 


MAIS(C) 


11.75 


MAIS(B-8) 


11.00 


Google 


9.50 


All-the-Web 


8.25 


Altavista 


6.50 



With 30 sites 


MAIS(C-int) 


19.50 


MAIS(A-55) 


18.25 


MAIS(C) 


16.25 


MAIS(B-8) 


15.00 


Google 


15.00 


All-the-Web 


13.50 


Altavista 


9.25 



6 Comparing MAIS and Other Multi-agent Based Tools 

Widyantoro and coworkers [26] discuss different representations of user profiles for 
personalization of information systems. The authors stress the importance of using 
long term and short term user profiles. They introduce a technique based on explicit 
user feedback, where users indicate positive and negative examples. Some other 
works are also based exclusively on user feedback: Amalthea [18], [3], Webnaut [20] 
and WAIR (Web Agents for Internet Research) [27], In such systems the user must 
provide a significant amount of feedback, increasing the human workload and the 
dependency of the system. The main difference with MAIS is how the user feedback 
is used for updating the user profile. 

In our approach we divide the user profile in n + 1 parts, where n is the number of 
interest centers selected by the user plus the user favorites model. For us, the user 
favorite model represents the long term model of the user profile. The short term 
model is built dynamically, when the user explicitly changes of interest centers select- 
ing new public centers of interest. We think that our approach reduces the need for 
user feedback because the user does not need to classify each document, but only to 
select some interest centers. In addition, the system behavior does not depend exclu- 
sively on the user feedback because the system can have already many feasible mod- 
els based on collections of high quality documents. 



7 Conclusions 

In this paper we analyzed the performance of MAIS [6] (Multi-Agent based Internet 
Search). The system is based on multi-agents systems, information retrieval and 
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automatic document classification techniques. It realizes meta-searches using 
standards search engines, personalizes searches and provides public interest centers, 
producing better results. Personalization on the web is a wide field. It seems that the 
idea of personalized multi-agent systems capable of realizing personalized tasks like 
search for information, information presentation, resources location (hotels, transport 
tickets and general products) becomes increasingly more realistic. In MAIS, personal- 
ization is carried out taking into account user favorite sites and public interest centers 
constructed previously and automatically from documents selected by experts. The 
performance of MAIS shows that public interest centers and favorite sites are an 
important source of information for improving the quality of the searches. 
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Abstract. Today’s software platforms that support the construction of agent sys- 
tems in accordance with the FIPA specifications essentially provide enabling in- 
frastructure services, and are still far away from adequately rendering support to 
current methodologies and theories for building agent systems, especially when 
social concepts play a significant role. Elsewhere, we have presented the 1ZTCA 
theory, that smoothly integrates relevant aspects of Agent Communication Lan- 
guages and Organisational Models, so as to provide guidelines for agent system 
design. This paper explores the impact of this theory on the actual development and 
implementation of agent-based applications. For this purpose, the 1ZTCA meta- 
model is considered as a programming language, in which roles, interactions, 
communicative actions, etc., are first-class language entities. We show how this 
language can be effectively implemented as a software framework that extends 
the JADE platform, and provide an example that illustrates its potential. 



1 Introduction 

In the last decade, the development of software agent applications has largely benefited 
from the standardisation efforts made by the Foundation for Intelligent Physical Agents 
(FIPA) [1], as well as from the many software platforms that implement the FIPA spec- 
ifications (e.g. [2, 3]). These platforms, however, provide rather basic infrastructure ser- 
vices, and are still far away from adequately rendering support to current methodologies 
and theories for building agent systems [4]. This is particularly true for approaches that set 
out from a social-level analysis [5], such as the Role/Interaction/Communicative Action 
(JZICA) theory [6, 7], which smoothly integrates relevant aspects of Agent Communi- 
cation Languages (ACL) [8, 9] and Organisational Models (OM) [10, 1 1 ] by introducing 
the concepts of Communicative Roles and Interactions, so as to provide guidelines for 
agent system specification. 

This paper explores the impact that the RICA metamodel has on the design and, 
in particular, on the implementation of agent-based applications: it shows how agent 
system programming can be leveraged by relying on roles and interactions as first- 
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class programming constructs, and by structuring these abstractions in terms of ACL 
components. The paper is organised as follows: Section 2 outlines the fundamentals of 
the RICA theory in terms of programming language metaphor. In Section 3, we present 
the architecture and design of the RICA-J prototype, a software framework implemented 
on top of the JADE platform [2] . Section 4 gives an example for the use of this framework, 
which illustrates how an important part of agent system programming gives rise to, and 
may rely on, highly-reusable libraries of communicative roles and interactions. Finally, 
we present the lessons learnt from this enterprise. 



2 The 1ZXC A. Theory 

The 1ZTCA theory provides a conceptual framework which establishes a systematic link 
between the specifications of the organisational model and the ACL of a multiagent appli- 
cation. According to the theory, the specification of the organisational model comprises 
the definition of entities such as agents, roles and interactions, while the specification of 
the ACL deals with the definition of communicative actions and protocols. The 1Z1CA 
theory is made up of two major components: a metamodel, which defines the language 
used to specify the different types of organisational and communicative entities; and a 
specification of the structure and behaviour of these entities. 

2.1 The 1ZICA. Metamodel 

The 1Z1CA metamodel is outlined by the UML class diagram of figure 1 . The relations 
between its entities are subject to a number of constraints which will be discussed in the 
sequel. 



content 




Fig. 1 . The 1ZICA metamodel 



The behaviour of agents manifests itself in the form of different roles. So, the defini- 
tion of a (type of) agent basically requires the specification of a collection of (types of) 
roles 1 . Role definitions required the specification of a collection of actions, which repre- 



1 In the sequel, the qualification “type of” will be omitted whenever the context makes clear that 
we are talking about types and not instances. 
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sent the specific tasks to be performed as part of the role’s functionality. Moreover, roles 
which delegate part or all of their functionality to other roles are called enclosing roles. 
The definition of an action requires, in turn, the specification of its types of input and 
output parameters. The 1Z1CA theory establishes a distinction between generic roles 
and so-called social roles, a kind of role which represents the functionality of agents 
participating in social interactions. The state of a social interaction is represented by a 
collection of parameters. 

The distinction between generic actions and, so-called, social actions , parallels the 
one established for roles. Input/output parameters defined for a social action must have 
been declared as state parameters of its corresponding type of interaction. In this way, the 
state of social interactions will be updated as a result of the execution of social actions. 
Communicative actions are conceived as special kinds of social action performed with 
the intention to be observed by agents playing a specific role [12]. The propositional 
content parameters of a communicative action simply consists of its input parameters. 
There are actually two parallel hierarchies of actions, corresponding to disjunctive and 
single actions, respectively. The FIPA inform-if and request communicative actions are 
examples of disjunctive and single communicative actions, respectively. In the context 
of an interaction, protocols are optional artefacts that prescribe the (social) actions that 
agents must perform. According to the 1Z1CA metamodel, a protocol must identify 
the type of regulated interaction (which, in turn, constrains the roles, actions, etc. that 
the protocol can refer to), and may be augmented by whatever protocol specification 
technique (AUML sequence diagrams, Petri-nets, interaction machines, etc.). 

The following example, which closely follows a similar one found in the JADE 
distrubtion, illustrates the use of the 1Z1CA concepts as a conceptual framework for 
modelling multiagent applications. Consider a simple e-commerce setting. Purchase 
interactions are established between pair of agents, playing respectively the roles of 
buyer and seller. The parameters declared for this type of interaction include the item to 
be purchased, the credit card used in the payment, its successful status, etc. Buyers may 
perform different FIPA communicative actions. For instance, they may request the seller 
to sell some item (a type of social action), or cancel the selling action once it has been 
initiated. Furthermore, sellers may evaluate (a type of social action) whether the sell 
action can be performed, and under which conditions the item should be sold. Sellers 
may also take communicative actions to agree or refuse to sell an item, etc. The FIPA 
Request protocol is used as a purchase protocol, client agents playing the role of buyers, 
and clerk agents act as sellers, both using FIPA ACUF1PA SL as their communication 
language. 

The remaining entities of the 1Z1CA metamodel in 1, communicative roles and com- 
municative interactions, establish the link between communicative and organisational 
concepts. The rationale behind these particular kinds of social roles and interactions re- 
lates to reusability matters: given that requests and cancellations, for instance, might be 
declared for other roles different from the buyer role, it makes sense to define a generic 
role, named requester, encapsulating the declaration of these actions. Then, the seller 
role, as well as any other role requiring these CAs as part of its characteristic behaviour, 
may simply reuse their declaration from the requester role. Similarly, the FIPA Request 
protocol may be attached to a generic social interaction which declares the interaction 




On the Impact of Agent Communication Languages 



95 



parameters and communicative roles required by the actions included in the protocol. 
Social roles that are defined in this manner (in terms of communicative actions) are 
called communicative roles. In a similar way, social interactions, defined by protocols 
and communicative roles, are called communicative interactions. 

Communicative roles and interactions encapsulate the whole pragmatic competence 
of agents. So, non-communicative social roles must directly or transitively specialise 
from, or delegate their communicative competence to, some communicative role. Other 
communicative entities in the e-commerce example may include the requestee com- 
municative role, encapsulating the declaration of the agree and refuse CAs, and action 
performing communicative interactions, which can be regulated, among others, by the 
FIPA Request protocol. The parameters declared for this type of interaction, include 
the action to be performed, the successful status of the interaction, etc. Note that com- 
municative roles may also define non-communicative social actions. For instance, the 
evaluation action identified for the seller role could actually be a social action declared 
for the generic requestee role. 

Figure 2 shows an instantiation of the 1Z1CA metamodel for the e-commerce appli- 
cation, using an specialisation pattern. The model is depicted as a UML class diagram, 
using stereotypes to indicate the type of each declared element. Parameters and the lan- 
guage spoken by agents are represented as attributes, while actions and social roles are 
modelled after UML operations and the interface classifier, respectively. Concerning 
the instances of the 1Z1CA relationships, they are represented using UML associations, 
except to model specialisations, where the UML generalisation is used instead. Note 
that the declaration of the action parameters, which would be represented by operation 
attributes, have been omitted from the diagram. 




Fig. 2. The organisational model of a simple e-commerce application 

2.2 Structure and Behaviour of TZICA Entities 

Figure 3 shows a static view of the relationships which hold between the different 
instances of the 1Z1CA entities at a given run-time slice. Agents, roles , social interactions, 
etc. are considered as the most generic types of entities, specifying the common structure 
and behaviour of its instances. Thus, they play the same role as the java class Ob j ect, 
which any programmed class has as super-class. 
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Fig. 3. Structure of 1ZTCA entities 



As shown in figure 3, at a given time, agents may be concurrently playing several 
roles 2 . A number of other roles, qualified as abandoned, may have previously been 
played. The predefined behaviour common to all agents is characterised by examining, 
whenever an agent is created or some event occurs, its collection of roles in order to 
identify abandoned roles that must be wake-up, and active roles that must be abandoned. 

Roles whose type declarations include the definition of some enclosing role set up a 
link to the latter at the time of their instantiation. When a role is played or abandoned, 
its characteristic behaviour is activated or deactivated, respectively. In general, the be- 
haviour of some role, which specifies the conditions that must be met in order to perform 
its declared actions, is defined by its type. However, pure enclosing roles (whose fun- 
cionality is completely split into other roles) have a predefined behaviour which defaults, 
similarly to agents, to periodically examining its enclosee roles in order to update the 
collections of active and abandoned roles of the player agent. Social roles for which 
some interaction type is defined, have a predefined behaviour as well. Besides examin- 
ing their enclosee roles, they monitor the conditions defined for initiating interactions of 
the declared type. In addition, they go through the ongoing interactions to see if some of 
them must be finished. At any time, an agent playing a social role, may be participating 
in multiple on-going interactions in accordance with the role’s type. Interactions can 
finish prematurely, because of some abnormal execution or by an explicit abortion, or 
upon completion (either successfully or unsuccessfully). 

The initiating conditions as well as the general agent behaviour within some inter- 
action is specified by its type declaration. However, those interactions regulated by a 
collection of protocols will delegate by default these responsibilities to them. Hence, 
protocol types are required to define their particular initiating conditions, as well as the 
logic required to identify which activity a given role must perform at any given time. The 
behaviour of a role in the context of an interaction regulated by some protocol defaults 
to performing the required activities until the protocol itself signals the completion of 
the interaction. The activities prescribed by protocols comprise the (concurrent or se- 
quential) performance, observation, or abortion of social actions. The performance of 
any kind of action (social or non-social) requires a collection of consistently instantiated 
input parameters, while its result is conveyed through the output parameters. Social ac- 



2 The types of these roles must obviously conform to the declaration of the agent type. This kind 
of type conformance applies throughout the following discussion. 
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tions are executed in the context of a particular interaction, in such a way that its input 
and output parameters values refer to the parameters representing the actual state of the 
interaction. In general, the particular activity performed by some agent when executing 
some action must be defined by its corresponding type. Communicative actions, though, 
have a predefined activity, which consists of sending a message to some agent’s role in 
any language understood by the receiver agent (e.g. FIPA ACL/FIPA SL). With regard to 
the observation of other agent’s actions, the particular way to achieve this activity must 
be defined by the corresponding type of social action. The same applies to the abortion 
of the execution of some action. 

Note that the 1Z1CA metamodel, together with the execution-based semantics just 
sketched, can be considered as a programming language offering different types of 
constructs of organisational and communicative nature. In fact, the declarations of the 
different types of entities identified by the 1Z1CA metamodel can be conveyed by pro- 
gramming language constructs, as an alternative representation mechanism to the UML 
modelling notation. Just as Java programs are composed of declarations of classes, meth- 
ods, attributes, etc., a 1Z1CA program comprises declarations of the different agents, 
roles, interactions, etc., required by the application. As an example, the following piece 
of code conveys the structure of the declaration of some role. 



social role type SomeRole [specialises BaseRole] 
activation{ 

plays Rolel, Role2, ... 

performs { 

action type actionl (in paraml, . . . , out paramN, . . . ) 

on execution{ . . . } 
on abort ion { . . . } 



In the example, reserved words are capitalised and relate to the different types of en- 
tities and relationships of the 1Z1CA theory. The syntax and indentation features attempt 
to resemble those of the Java language. Any conventional programming language (e.g., 
Java) may complete this organisational-based agent programming language, by provid- 
ing the specification of the activation conditions and the execution code of their actions. 



3 7 ZXCA—J: A 1ZXCA - Based Software Framework 

This section describes the VJICA—J software framework (an acronym for 7?.XC_4-.Iade), 
which allows to program multiagent systems based on the 1Z1CA theory and its oper- 
ational semantics that we have sketched in the previous section. We have chosen to 
develop the software framework on top of an existing tool, in order to reuse those mid- 
dleware aspects required for supporting agent interactions, as well as the different agent 
abstractions provided. In particular, we use Jade [2], a FIPA-compliant platform imple- 
mented in Java, so our framework is also based that language. The implementation of 
the 1Z1CA layer (implemented in a java package named rica) was designed with the 
goal of relieving, as much as possible, the application developers from those program- 
ming tasks that could be automated. These tasks include, for instance, the programming 
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of the agent’s and generic role’s predefined behaviours, dealing with the instantiation, 
playing and abandonment of roles; the programming of the default behaviours of agents 
participating in some interaction under some protocol; the updates on the interaction 
state resulting from the execution of actions, etc. 

In order to achieve this goal, the types of the application agents, roles and interactions, 
etc., must be available at runtime. Hence, the 1Z1CA layer has been subdivided into two 
modules: A reflective module, implemented by the package rica .reflect (an analogue 
of the java, reflect standard package), containing those classes which implement 
the metamodel entities of the 1Z1CA theory: AgentType, RoleType, etc.; and a core 
module, implemented by the package rica. core (an analogue of the jade, core 
package), providing classes implementing the structure and common behaviour of the 
different 1Z1CA abstractions: Agent, Role, Sociallnteraction, Protocol, etc. 
The next subsections will describe the structure and design guidelines followed in the 
implementation of the reflective and core modules. 

3.1 The reflective Module 

The rica . reflect package provides an implementation of the metamodel entities of 
the 1Z1CA theory, in such a way that the class diagram of figure 1 , conveys the general 
structure of this package as well. AgentType, RoleType, etc., classes are analogues to 
the java class Class, which represent the type of some declared object. The Object 
class, in turn, is an analogue of the classes implemented in the rica. core package, 
which are objects of a more specific nature. The basic problems faced in the design of 
the reflective capabilities of the framework are related to these questions: 

- Firstly, how can the declaration of the types of roles, interactions, and so on, be 
embedded in the declaration of java classes? 

- Secondly, how can these declarations be accessed? 

The devised solution to answer the second question, which sets up the premises to 
solve the first one, follows the pattern used by the Java language itself. In java, type 
information is accessible in two ways: statically, using the static class field, automat- 
ically declared for any class, and, dynamically (at run-time), by the Object instance 
method, getClass ( ) . Similarly to the class field, any class specifying some 1Z1CA 
type will necessarily declare a static field, named type, containing the declaration of 
its type. Concerning the dynamic access to the type of some 1Z1CA object, getType ( ) 
instance methods will be defined for each of the 1Z1CA top-level classes: Agent, Role, 
Sociallnteraction, etc. This methods will access the corresponding type field de- 
clared for the dynamic class of the object. Figure 4 illustrates this design pattern using 
some role entity as example. 

With regard to the first question, the implementation of an entity type will be com- 
monly carried out by means of the implementation of a java class 3 . In those cases, the 
declaration of the type will be embedded in the class by means of public static 
final fields, which will be, in general, initialised by accessing the type fields of the 



3 The exception are parameter types which are implemented as fields of the declaring classes of 
interactions. 
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Fig. 4. Reflective design pattern 



declaration members 4 . Then, the declaration will be created automatically by accessing 
these static fields. As shown in figure 4, the classes implementing the 1Z1CA metamodel 
entities, must declare a constructor which takes as parameter the java class corresponding 
to the static class of the 7 Z1CA entity. 

All the java classes implementing some entity type will follow the same implemen- 
tation schema corresponding to that type. For instance, the schema for embedding the 
declaration of a role type in a java class is as follows: 



public [abstract] class SomeRole extends BaseRole | Role 
{ 

/** Static type. */ 

public static final RoleType type = new RoleType (SomeRole . class ) ; 

/** Played roles. */ 

public static final RoleType rolel = SomeRolel . type; 
public static final RoleType role2 = SomeRole2 . type; 



/** Performed actions. */ 

public static final ActionType actionl = SomeActionl . type; 
public static final ActionType action2 = SomeAction2 . type; 



3.2 The core Module 

The rica.core package is designed around the collection of classes shown in figure 
5. These classes implement the major types of TZ1CA entities whose structure and 
dynamic behaviour was described in section 2.2. As can be observed, the abstract classes 
(interfaces) of the diagram represent the base action types. Single and disjunctive action 
hierarchies provide actual implementations of these interfaces (not shown in the figure). 
On the other hand, the only TZ1CA entities which are left from the diagram of figure 5 
are: parameters, which, as was previously explained, are embedded in java as fields of 
the corresponding interaction and action classes, and languages, whose implementation 
will be described bellow. Also, note that social interactions objects in the HICA—J 
framework are not shared by the different participating agents, but are owned by a 
single participant’s role. In this way, social interaction objects represent the view of 



4 Thus, the static accessor will be used for declaration purposes, while the getType ( ) instance 
methods will be used at run-time. 
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the interaction from one participant’s perspective, called its (characteristic) performer. 
Later, we will deal with the issue of participants representation. 




Fig. 5. Major structure of the rica . core package 



Figure 5 also shows the main methods declared by these classes, which manifest a 
close correspondence with the computational semantics prescribed for each type of entity 
(methods parameter types are omitted from the class diagram to improve its readability). 
We can distinguish between two different kind of methods: those corresponding to the 
dynamic characteristic of the declared types, and those related to the state of its instances: 

- Type Related Methods. As it was previously described, agent, role, social role, and 
action types, require certain characteristics concerning its functionality, which com- 
plement the static characteristics identified by the 1Z1CA metamodel. For instance, 
role types must specify its characteristic behaviour and activation conditions. These 
components of their declaration can be specified by overriding the setBehaviour, 
shouldBeAbandoned and shouldBePlayed methods. Similarly, the setlnter- 
actionBehaviour and shouldlnitiatelnteraction and methods allow par- 
ticular social roles to specify their conditions to initiate its participation in some 
interaction, as well as its characteristic behaviour within it. Moreover, programmers 
can implement the handlelnitiatedlnteraction and handleFinishedln- 
teraction abstract methods, which are hooks invoked when the specified interac- 
tion is initiated or finished, respectively. Some of these type of methods have prede- 
fined implementations, as those previously mentioned for social roles, which resort to 
the protocols declared for its social interaction type, or those related to the behaviours 
of agents and pure enclosing roles. Other ones, such as those corresponding to the ac- 
tivation conditions of roles, or the execution functionality of actions are declared as 
abstract. Finally, some of them are declared as final, as those fixing the general 
behaviour of social roles, or the execution functionality of communicative actions. 
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- Instance Related Methods. These methods do not carry type information, but sim- 
ply allow to change or observe the state of the different instances. In this way, the 
play (RoleType r) and abandon (RoleType r) methods, cause some agent to 
play and abandon the role instance of the specified type (these methods will be 
called from the predefined behaviour of the agent), and the as (RoleType r) :Role 
method returns the role instance of the specified type currently being played by 
the receiving agent. Similarly, the launch ( ) method starts the execution of some 
action, and the initiate (Sociallnteraction i) method cause the specified 
interaction in which some social role is participating to be initiated. Last, the get- 
NextActivity ( ) method of the Protocol class, allow the social roles to get the 
activities that must be performed in each step of some interaction according to the 
state of the protocol. 

The behaviours of agents and roles are represented by means of the Jade abstrac- 
tion, jade. core. Behaviour. The predefined behaviour for agent types is imple- 
mented as a cyclic behaviour, which checks the activation conditions of the agent 
roles, through the shouldBeActivated method. Once this method returns true, the 
role’s characteristic behaviour will be scheduled for execution. In the case of social 
roles, its predefined behaviour is a parallel behaviour which necessarily includes a 
cyclic behaviour which invokes the shouldinitiateinteraction method, to check 
if some interaction must be initiated. The default implementation of this method cre- 
ates an instance of the social role’s interaction type, and checks if any of the pro- 
tocols that regulate its type of interaction (which will be accessed using the reflec- 
tive capabilities, i.e. the getType ( ) method) signals the beginning of the interaction. 
As soon as the shouldBeinitiated protocol’s method returns true, the interaction 
will be initiated, and the social role’s interaction behaviour (the one returns by the 
get Interac tionBehaviour method), will be scheduled as part of the social role’s par- 
allel behaviour. Moreover, the handlelnitiatedlnteraction method would be in- 
voked. The social role’s predefined interaction behaviour performs the activities returned 
by the getNextActivity method of the protocol (execute, observe or interrupt some ac- 
tion), until the end of the interaction, which will be signalled by its hasBeenCompleted 
method. In this way, the whole dynamics of some agent concerning those aspects re- 
lated to the management of interactions (its initiation, development and finalisation) are 
automated through the predefined behaviours of agents and social roles. 

The rica . core package also includes other classes providing the glue linking the 
1Z1CA abstractions with those provided by the Jade platform. Among these classes, 
the ACLMessageTranslator class helps to translate some communicative action to 
and from an object of the jade . lang. acl . ACLMessage class. To achieve this task, 
the language spoken by the agent performing the communicative action (represented 
by the Jade abstraction, jade . content . lang . Codec), and the ontology of the so- 
cial interaction in which the action is performed (represented by an instance of the 
j ade . content . onto . Ontology class), will be used. The rica . core . RolePlay- 
ingDFService bridge class provides convenient ways to published the types of social 
roles that some agent is currently playing, as well as its spoken languages, through the 
j ade . domain . DFService class. In this way, when some agent is committed to initi- 
ate an interaction and must look for other participants, agents playing the types of roles 
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declared for the interaction’s type, and speaking some of the agent’s languages, can be 
searched for in the directory service. 



4 Programming in 1ZICA—J' 

The design of a multiagent application in the 1ZICA—J framework departs from its 
organisational analysis in terms of the 1Z1CA metamodel, possibly elaborated on the 
basis of the method suggested in [6]. This analysis will uncover those communicative 
interactions that application-dependent agents and interactions rely on. In case that a need 
for the use of protocols is identified, the 1Z1CA model of the application is complemented 
with their proper specification, using for that purpose whatever protocol specification 
technique be most adequate. 

The architecture of application programs in the 1ZICA—J framework is shown in 
figure 6. Applications rely on the reuse of two kinds of component libraries: those im- 
plementing the communicative aspects of the application-dependent interactions, and 
those allowing the implementation of protocol specifications on the basis of different 
kinds of formalisms. These libraries can be based upon FIPA specifications or other 
contributions made available by independent application developers. Currently, most 
of the FIPA ACL components are implemented in a library of communicative interac- 
tions under the fipa java root package: fipa.closedActionPerforming, fipa. 
closedinf oExchange, etc. Protocols are implemented based on so-called interaction 
machines [7], available through the protocol . interactionMachine package. 



M ULHAG ENT APPLICATION 



application-dependent 
organisational components 




Lj i i. 

RicaJ 



Fig. 6. Architecture of a 1Z1CA—J application 

Given that the necessary protocol specification and instrumentation techniques are 
available, the implementation of a multiagent application based on 1ZICA—J requires 
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to design and implement (1) the communicative interactions needed that are not present 
in the available communicative libraries, and (2) the application-dependent interactions 
and agents. In the following we will illustrate both steps on the basis of the e-commerce 
model of figure 2, and using interaction machines as protocol specification formalism. 

On the one hand, the implementation of communicative interaction types and their 
corresponding communicative roles, actions, protocols, etc., components, requires em- 
bedding their 1Z1CA definitions in proper subclasses of the VJICA—J core superclasses, 
as explained in section 3.1. On the other hand, for fully implementing the different types 
of components, additional aspects concerning the behaviour of these entities must be 
addressed, following the general requirements described in section 3.2. This section de- 
scribes the particular way in which communicative and social components will generally 
addressed these implementation features. For instance, the playing and abandonment 
conditions for communicative roles, which are implemented by overriding the corre- 
sponding shouldBePlayed( ) and shouldBeAbandon ( ) Role methods, should be 
left unspecified, as these conditions closely depends on the specific type of social role spe- 
cialising the communicative role. Similarly, the handlelnitiatedlnteraction ( ) 
and handleFinishedlnteraction ( ) , should not be implemented. On the contrary, 
those aspects concerning the conditions that should be met in order to initiate and interac- 
tion of the kind declared for the communicative role, will be usually defined at this level. 

For instance, an action performing interaction is initiated when the requester gener- 
ates the intention that some action be performed, and some requestee partner has been 
found. Following this scheme, the implementation of the ActionPerforming class, 
which is partially shown in table 1, provides a constructor for the requester role which 
updates its action parameter, taking into account the intentional state of the specified 
role. If some action shall be performed, a partner agent playing the requestee role is 
located through the role-based directory service 5 . 

The action performing interaction is instantiated by the requester agent once the 
monitoring behaviour of its social role’s behaviour is restarted, and the shouldini tia- 
teinteraction method is subsequently invoked. Then, the fipa request protocol is 
instantiated and asked whether the requester must initiate its participation in the interac- 
tion. This protocol, implemented by means of an interaction state machine, is initiated if 
the initial request transition can be activated, which in turn, requires the action parameter 
(its propositional content) to be initialised. Hence, the interaction will be initiated if this 
parameter was set in its instantiation. 

The implementation of the Requester role provides the getlntendedAction 
method, as well as the setlntendedAction and droplntention, to update its in- 
tentional state. These methods may only be invoked from the user interface of the agent, 
or from other roles which the agent is currently playing. Moreover, they will cause the 
role’s behaviour to be restarted, in order to initiate a new interaction, or finish an ongoing 
one through the cancel communicative action. 

Besides communicative actions, just as the Request and Cancel actions declared by 
the requester agent, non-communicative social actions can be defined for communicative 



5 The setRestOf Participant method, declared by the SocialRole class, achieve this 
task, thanks to the reflective capabilities of the framework. 
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Table 1 . Implementation of the action performing interaction type 



public class ActionPerforming extends Communication { 

/** Type info. */ 

public static final ParameterType action = new ParameterType ( "action" , SocialAction. class) ; 
public static final CommRoleType requester = Requester . type; 
public static final CommRoleType requestee = Reques tee . type; 

public static final ProtocolType submiss ionProcedure = FIPARequestProtocol . type; 



/** Constructor for requesters . */ 
public ActionPerforming (Requester requester) { 
super (requester) ; 

setAction ( requester. getlntendedAction ( ) ) ; 
if (getAction ( ) ! =null) { setRestOf Participants () ; ... } 

} 



public class FIPARequestProtocol extends InteractionMachineProtocol{ 

/** Type info. */ 

public static final CommlnteractionType interaction = ActionPerforming. type; 

/** Interaction Machine */ 

public static class FIPARequestMachine extends Interact ionMachine{ 

/** States. */ 

public static final InitialState initial = new InitialState ( "initial" ) ; 

public static final State evaluating = new SimpleState ( "evaluating" , Evaluate . type) ; 

public static final State performing = new SimpleState ( "performing" , action); 



/** Transitions . */ 

public static final Transition initial_evaluating_request = 

new Transition ( "initial_evaluating_request" , initial , evaluating. Request . type) ; 
public static final Transition evaluating peforminq refuse = 

new Transition ( "evaluating_performing_refuse" , evaluating, done_unsuccessfull , Refuse . type, 
new Not (new Successfull (Evaluate . type) ) ) ; 

} 

} 



public abstract class Requester extends Communicator { 

/** Type info. */ 

public static final CommActType request = Request . type ; 
public static final CommActType cancel = Cancel. type; 



/** Intention management. */ 
private SocialAction intendedAction; 

public void setlntendedAction ( SocialAction action)! ...; getRoleBehaviour () .restart (); } 
public SocialAction getlntendedAction () { ...; return action}; 

public void dropIntentedAction (SocialAction action) {. . . ; getRoleBehaviour ( ) . restart () }; 



public abstract class Requestee extends Communicator! 

/** Type info. */ 

public static final CommRoleType type = new CommRoleType (Requestee . class) ; 
public static final CommlnteractionType interaction = ActionPerforming. type; 
public static final CommActType agree = Agree. type; 
public static final SocialActionType evaluate = Evaluate . type ; 



/** Called from Evaluate */ 

public abstract boolean willPerform( SocialAction action) ; 
public boolean willlnmediately (SocialAction action)! return true;} 



public class Seller extends Requestee! 

/** Type info. */ 

public static final ActionType sell = Sell. type; 



/** Overriden Requestee interface. */ 

public boolean willPerform( SocialAction action) ! 

return (action instanceof Sell) && isValidCreditCard( ( (Sell)action) .getCreditCard( ) ) ; 

} 

/** Overriden role object interface. */ 
public boolean shouldBePlayed ( )(...} 
public boolean shouldBeAbandoned ( )!...} 



role types as well, as for the requestee role. However, the full specification of these types 
of actions will normally be available at the level of application-dependent roles. Thus, 
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the particular implementation of the evaluate action performed by this type of roles, 
will depend on the kind of social role reusing its declaration. The evaluate action can 
be implemented by the Evaluate class, a subclass the SingleSocialAction core 
class. Its execute method could be left unspecified, but an alternative implementation 
schema, resorting to abstract methods of the Requestee class is more convenient. 
This class provides implementations for identifying as to whether the action must be 
performed, and if it will be performed immediately: 

Finally, application-dependent components typically consist of social roles and in- 
teractions. As the pragmatic competence of social roles is implemented by the commu- 
nicative roles that they specialise, the only thing left to be determined is the moment in 
which this competence will be performed, i.e. its activation conditions, and the possi- 
ble actions performed in the beginning and ending of the interactions. For instance, the 
buyer role, as a kind of requester, can be implemented, as shown below, by extending 
the Requester class described above. Its activation is set by the interval of time during 
which its human counterpart is connected to the e-commerce portal or some purchase 
is pending to be completed. The activities performed in the initiation and finishing of 
some purchase will most probably be related to updates in its user interface. 



5 Conclusions 

In this paper we have shown how multiagent applications can be developed based on the 
reuse of communicative components, structured along organisational concepts, which 
encapsulate the ACL of the MAS. In particular, we have presented the 1ZICA—J soft- 
ware framework that proves that communicative roles and interactions cannot only be 
used as conceptual tools for the analysis and design of ACLs (see [6] for an analysis 
of FIPA ACL in these terms), but also as software components providing much of the 
functionality that application-dependent social roles and interactions require. 

The IZICA—J framework allows to build up highly-reusable libraries of commu- 
nicative roles, interactions and protocols, and thus relieves MAS programmers of most 
tasks associated with the management of interactions, a key issue in the implementation 
of any MAS. Moreover, it is based on a separation of concerns, as the pragmatic compe- 
tence of agents is encapsulated in special software modules, and is thus kept away from 
non-communicative aspects of the application. 

The framework also stresses the use of interactions as first-class programming ab- 
stractions, something which is absent from JADE and other MAS frameworks. A no- 
table exception to this rule are approaches based on electronic institutions [13]. Still, the 
IZICA—J framework relies on a programming language metaphor and a specification of 
the execution-based semantics of the different types of organisational meta-entities. This 
bias towards organisational modelling also differentiates our approach from other pro- 
posals in the field of agent programming languages, which tend to focus on intra-agent 
programming matters, rather than on the social structure of the application. 

Future work comprises the (re-)implementation of several MAS applications so as to 
fully explore the potential and limitationsuse of the IZICA—J framework. Furthermore, 
we are interested in relating our work to current FIPA efforts in the field. 
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Abstract. Given a specification of communication rules in a multiagent system 
(in the form of protocols, ACL semantics, etc.), the question of how to design 
appropriate agents that can operate on such a specification is a very important 
one. In open systems , the problem is complicated even further by the fact that 
adherence to such a supposedly agreed specification cannot be ensured on the side 
of other agents. 

In this paper, we present an architecture for dealing with communication pat- 
terns that encompass both a surface structure of admissible message sequences as 
well as logical constraints for their application. This architecture is based on the 
InFFrA social reasoning framework and the concept of interaction frames. It as- 
sumes an empirical semantics standpoint by which the meaning of communication 
is pragmatically interpreted through decision-theoretic optimality considerations 
of a reasoning agent. We introduce the abstract architecture and a formal model and 
present experimental results from a complex domain to illustrate its usefulness. 



1 Introduction 

The process of designing agent communication languages, interaction protocols and 
conversation policies is primarily concerned with what goes on between rather than inside 
agents. It is therefore only natural that ACL research has rarely attacked the problem of 
how to build agents in accordance with a given set of semantic rules, protocol structures, 
etc. 

In the light of open multiagent systems, in which agents are not controlled by a cen- 
tral entity or pursue common goals and need not be benevolent, this design problem is 
complicated even further. Not knowing how other agents will apply a given specification 
concerning the semantic and pragmatic properties of the provided means of communi- 
cation, the question for an agent is not only how to apply it by himself, but also whether 
to comply with the supposedly agreed specification at all. 

This is exactly the issue this paper deals with: Given the specification of a number of 
communication patterns together with logical constraints for their application, how can 
we design an agent that uses them appropriately to further his own goals, not knowing 
whether other agents will comply? As a possible answer to this question, we propose an 
agent architecture based on the concept of empirical semantics [10]. This architecture 
can be seen as a “communication reasoning” component that takes as inputs the belief 
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state and utility function of an agent as well as a set of models of communication patterns 
(so-called interaction frames ) and outputs communication decisions. The architecture 
is characterised by two essential features: 

1 . It uses decision-theoretic principles to make optimal action decisions while taking 
previous communication experience into account. 

2. It combines empirical knowledge with a priori specified logical constraints for 
communication patterns. 

Most importantly, the reasoning heuristics used in this architecture are computation- 
ally tractable and can be implemented directly. We thus contribute to the task of engineer- 
ing agents that are able to operate flexibly on a given set of communication protocols and 
semantic rules and under the assumption that the empirical semantics view is followed. 

The remainder of this paper is structured as follows: In section 2 we review the foun- 
dations of the empirical semantics approach. Subsequently, we introduce the InFFrA 
social reasoning framework that the proposed architecture is based on. Section 4 estab- 
lishes a formal model of empirical semantics called rrflnFFrA that uses InFFrA frames 
to capture probabilistic expectations about communication. In section 5 we then present 
the apparatus necessary for making decisions in this model. Experimental results from 
a concrete implementation of the model in a complex domain are given in section 6 to 
underline its practical usefulness. Section 7 rounds up with some closing remarks and 
conclusions as well as an outlook to future work. 



2 Empirical Semantics 

In [10], we proposed a new model for communication based on empirical semantics. 
The basic idea behind this model is that communication obtains its meaning through 
the interactions within which it occurs. The central instrument to establish this meaning 
are the expectations constructed by agents participating in or observing an interaction. 
These expectations “contain” the current semantics of communication in a given social 
context and their evolution over time mirrors the evolution of meaning. 

To make things a bit more concrete, we quote the following central elements of the 
empirical semantics approach from [10]: 

1 . The meaning of a message can only be defined in terms of its consequences, i.e. the 
messages and actions that are likely to follow it. Two levels of effects can be distin- 
guished: 

(a) The immediate reactions of other agents and oneself to the message. 

(b) The “second-order” impact of the message on the expectation structures of any 
observer, i.e. the way the utterance alters the causal model of communicative 
behaviour. 

2. Any knowledge about the effects of messages must be derived from empirical obser- 
vation. In particular, a semantics of protocols cannot be established without taking 
into account how the protocols are used in practice, 

3. Meaning can only be constructed through the eyes of an agent involved in the 
interaction, it strongly relies on relating the ongoing communication to the agent’s 
own goals. 
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Fig. 1 . Empirical semantics and its evolution 



In other words, our view is that the semantics of communication can be reduced 
to its pragmatics as perceived by agents who are using communication. While this is 
an overtly restrictive interpretation of meaning for general communication, it can be 
very useful for practical agent reasoning in open multiagent systems, in which “nor- 
mative” semantics (e.g. commitment-based speech act semantics [12]) may be proven 
wrong by others’ inability or unwillingness to comply with them. This is because of 
the fact that it can be used as a minimal-assumption method for reasoning about com- 
munication which at least ensures that the agent will not compromise his own welfare 
for the sake of complying with a pre-specified semantics (when others do not comply, 
either). 

Generally speaking, if the meaning of an utterance (or sequence of utterances) lies in 
the expected consequences, what is needed to capture semantics is a model of causality 
and correlation that allows to predict these consequences for a given (sequence of) 
utterance(s). This model will then be adapted with new observations so as to enable the 
agent to extrapolate past correlations into the future. 

Expectation networks [6, 7] have been suggested as a very general method of cap- 
turing this kind of expectations that relies on a probabilistic model of “continuations” 
between interrelated messages derived from statistical observation. Instead of going into 
the details of this formalism, we sketch the basic idea using the illustrations of figure 1 . 
In this figure, nodes represent communicative actions, edges correlations between them 
(variable line width is used to indicate different degrees of correlation). The shaded 
node sequence is used to describe the recently observed portion of the network, e.g. an 
ongoing conversation. The decreasingly dark shaded regions of predicted future actions 
denote that predictions regarding “distant” events are increasingly vague. The transition 
from the situation shown on the left to that on the right occurs upon observation of a 
new action that is appended to the currently relevant path. With this new observation, the 
correlation between the message previously observed and the current message increases 
compared to alternatives that did not occur. 

Taking the standpoint of empirical semantics enables an agent to adapt his expec- 
tations with new communication experience and to adjust his own behaviour to the 
expected reactions of others. However, this conceptual view of semantics does not say 
anything about how agents can use such expectations in practice. 
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3 The I n F FrA Architecture 

The Interaction Frames and Framing Architecture InFFrA [11] has originally been de- 
veloped as a meta-framework for social reasoning architectures based on the notions of 
“interaction frames” and “framing”. 

The central idea behind InFFrA is to employ models of classes of interaction called 
interaction frames to guide agents’ social behaviour. The process of applying frames 
appropriately in interaction situations is referred to as framing. In the abstract architecture 
InFFrA, a frame is a data structure that contains information about 

- the possible courses of interaction (so-called trajectories ) characteristic to a partic- 
ular frame, 

- roles and relationships between the parties involved in an interaction of this class, 

- contexts within which the interaction may take place, and 

- beliefs, i.e. epistemic states of the interacting parties. 

In computational terms, the trajectory model is usually a representation of a set 
of admissible message and action sequences, while the latter three elements can be 
collapsed into a single set of logical constraints which then have to be verified using the 
agent’s internal belief state (usually represented by the contents of a knowledge base). 

InFFrA makes use of a number of frame-based data structures to conduct the steps 
necessary for framing: 

- the active frame, the unique frame currently activated to describe the expected course 
of events, 

- the perceived frame, an interpretation of the currently observed state of affairs, 

- the difference model containing the differences between perceived frame and active 
frame, 

- the trial frame, used when alternatives to the current frame are sought for, 

- and the frame repository, in which the agent locally stores its frame knowledge. 

Using these data structures, an InFFrA agent performs the following steps in each 
reasoning cycle: 

1. Interpretation & Matching: Update the perceived frame and compare it with the 
active frame. 

2. Assessment: Assess the usability of the active frame in terms of 

(i) adequacy (compliance of frame conditions with the current situation), 

(ii) validity (the degree to which the active frame’s trajectory matches the perceived 
encounter) and 

(iii) desirability (depending on whether the implications of the frame correspond to 
the agent’s private goals). 

3. Framing decision: If the active frame seems appropriate, continue with 5. Else, 
proceed with 4 to find suitable alternatives. 

4. Adjustment/Re -framing: Search the frame repository for better frames. “Mock- 
activate” them as trial frames iteratively and go back to 1; if no suitable frame 
is found, end the encounter. 

5. Enactment: Derive action decisions by applying the active frame. 
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Fig. 2. Overview of the framing process 



This entire (simplified) framing process is depicted in figure 2. It should be 
emphasised that InFFrA only describes the social layer of agent reasoning; in order to 
obtain an integrated agent architecture, it has to be combined with a suitable component 
for local rational reasoning (e.g. a BDI [8] reasoner), so that the agent’s private goals 
and preferences can be taken into consideration during the assessment phase. Looking 
at InFFrA from an empirical semantics point of view, we can re-interpret the frames in 
an agent’s repository as the expectations about communication patterns that the agent 
holds. This provides us with a reasoning scheme for processing expectations which is 
missing in the pure “expectation-modelling” view of expectation networks laid out in the 
previous section. Moreover, if we ensure that frame conceptions are adapted with new 
experience, InFFrA is ideally suited for applying pre-specified communication patterns 
(e.g. traces of protocol execution as envisioned by the designer) in a goal-oriented fash- 
ion while making sure that others’ (or one’s own) deviance from these normative patterns 
influences the expectations about validity and relevance of these patterns in the future. 



4 A Formal Model of Frame-Based Empirical Semantics 

To obtain a formal computational model for representing and reasoning about com- 
municative expectations based on InFFrA, we have developed rrflnFFrA, a simple yet 
expressive model for describing two-party, discrete, turn-taking encounters which can 
be thought of as conversations between two agents, m InFFrA uses a sequence of message 
patterns (i.e. messages containing variables) as its trajectory model specifying the surface 
structure of the encounters described by a particular frame, together with a list of substitu- 
tions to capture concrete values of these variables in previously experienced interactions. 
Each substitution also corresponds to a set of logical conditions that were required for 
and/or precipitated by execution of the trajectory in the respective encounter. Finally, tra- 
jectory occurrence and substitution occurrence counters record the frequency with which 
the frame has occurred in the past. Formally, a frame in m InFFrA is defined as follows: 
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Definition 1. A frame is a tuple F = (T, 0, C. hr , he), where 

- T = (pi , P 2 , . . . ,p n ) is a sequence of message patterns pi £ AT the trajectory of 
the frame, 

- 0 = (i?i, . . . , $ m ) is an ordered list of variable substitutions, 

- C = (ci, . . . , Cm) is an ordered list of condition sets, such that Cj £ 2 C is the 
condition set relevant under substitution 't)j, 

- hr G is a trajectory occurrence counter list counting the occurrence of each 
prefix of the trajectory T in previous encounters, and 

- Hq £ is a substitution occurrence counter list counting the occurrence of each 

member of the substitution list 0 in previous encounters. 

In this definition, At is a language of speech-act [1] like message and action patterns 
of the form perf ( A, B, X ) ordo(A, Ac). In the case of messages (i.e. exchanged textual 
signals), perf is a performative symbol (request, inform, etc.), A and B are agent 
identifiers or agent variables and X is the propositional content of the message taken 
from a logical language C. In the case of physical actions (i.e. actions that manipulate 
the physical environment) with the special “performative” do, Ac is the action executed 
by A (a physical action has no recipient as it is assumed to be observable by any agent in 
the system). Both X and Ac may contain non-logical substitution variables that are used 
for generalisation purposes (as opposed to logical “content variables” used by agents to 
indicate quantification or to ask for a valid binding) . We further use A4 e C A4 to denote 
the language of “concrete” messages that agents use in communication (and that do not 
contain variables other than “content variables”). 

To illustrate these concepts and to explain the semantics of a frame, we will consider 
an example of how the FIPA contract net protocol [4] can be implemented in rrflnFFrA. 
For ease of presentation, we will write T(F), C(F), etc. to denote the respective el- 
ements of a frame F and use the compact notation ( Th(F ), C(F), 0h(F)) instead of 
(T, C,0,hT,he), where 



Th(F) 



h T {F)[ 1 ] h T (F)[ 2 ] h T (F)[n] 

> Pi > Vi'" » Pn 



and 



0 h (F)[i\ 0 (F )[i\. 



Table 1 shows an interaction frame for the success path of the contract net protocol, 
the following should be noted about this example: 

- As can be seen, the first condition set (corresponding to the empty substitution) 
contains feasibility preconditions of the respective performatives (in FIPA-SL [3] 
with additional timestamps of the form @i indicating at which trajectory step a 
condition has to hold), as far as they are relevant for frame execution. 

- Definition 1 does not constrain which conditions are to be stored for a specific 
enactment of a frame (even allowing for empty condition sets), and the task of 
extracting relevant or even crucial information from the agent’s knowledge is clearly 
nontrivial. However, the reasoning framework to be defined in the following section 
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Table 1. Interaction frame for the success path of the FIPA contract net protocol 

F cn = / ( cfp(Ai, A 2 , (R,P))) propose(A 2 , Ai, Q) 

i accept-proposal(Ai, Ai, Q) i do(A 2 , Ai, R)), 

({lX(P = Q) = Y, 

-iBref Al (any X lA 2 Done(R, P)) A -> BA 1 lA 2 Done(R ) @1, 
BA 2 lA 2 Done(R,Q) @2, 

BA 1 lA 1 Done(R, Q) A BA 1 lA 2 Done(R , Q) @3, 

Ba 2 Q @4}, {}, { damaged (tires)}) , 

<-[] 

—> [Ai / agent 1 , Ai / agent 2 , Pj price Of (tire, X), 

Q / priceOf (tire, 75), R/ sell(tire, 4)], 

—> [A\/ agent 3 , Ai / agent 1 , P/ price Of (tire, A'), 

Q / priceOf (tires , 400), R/replace(tires)]) V 



primarily uses conditions to identify similarities in encounters, while expectation 
is drawn from utilities obtained during frame execution, making this approach less 
sensitive to the way conditions are selected. 

As for the different failure cases covered by the contract net protocol, these could 
either be modelled implicitly by using timeouts and interpreting “silence as disapproval” 
or explicitly by virtue of additional frames. The frame of table 2, for example, models 
the refusal to submit a proposal. 

Together, F cn and F cr capture the following observations about previous encounters: 

- Five encounters started with a message matching cfp(A 1 , A^, (R, P)), three of 
them continued with a proposal by Ai, the other two with a refusal. In two of the 
former three cases, the proposal was accepted by A\ and the respective physical 
action carried out by A^. 

- One encounter has terminated after the second message or was continued with a 
message not matching accept-proposal(Ai, Ai, Q). 

- For four of the five encounters, substitutions are available. Two contracts were 
made, one about the delivery of four tires for a price of 75 each, one about the 
replacement of a set of tires for 400. Two calls for proposals for the delivery of four 
tires were refused because no tires were available (which is a generalisation over 
two encounters, leaving Ai unspecified). 

To use m InFFrA frames for the prediction of future encounters, however, we need a 
formal model of their prospective rather than retrospective semantics. Here, the idea is 
(as in [10]) to use an entire repository T = {Fi, . . . , F. n } of frames, each representing a 
set of message/action sequences by virtue of the substitutions that can still be applied to 
its trajectory. Given an encounter prefix, i.e. a sequence of messages already uttered in the 
current encounter, and the agent’s current belief state, we can filter out those paths that 
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Table 2. Interaction frame modelling the refusal to submit a proposal in the FIPA contract net 
protocol 

F cr = (( ~> cfp(Ai, A 2 , ( R,P )) ^ refuse(A 2 , Ai, Q)), 

({-i Bref Al (any X lA 2 Done(R, P)) A -> BA 1 lA 2 Done(R ) @1, 
BA 2 (^Feasible(R) A Q A -> Done(R) A -> lA 2 Done(R )) @2}, 

{Ba x (inStock (tire)) @1}), 

(-[] 

[A\ / agents, P/priceOf (tire, X), 

Q / -<inStock (tire ) , R/ sell(tire, 4)] 



either (a) do not match the encounter prefix or (b) are labelled with logical conditions not 
satisfied under current knowledge base content. Considering the remaining (i.e. relevant) 
paths, we can then assign probabilities to all the possible encounter continuations (or 
postfixes) using the counter values of F £ T. Additionally, domain-dependent case- 
based reasoning [5] techniques can be applied by introducing a similarity measure on 
messages, such that different instances of a postfix pattern have different probabilities 
depending on their similarity with past enactments of the respective frame. Hence, T 
represents a simplified version of an expectation network that has the form of a tree. 

More formally, let w £ M* the encounter prefix and 

Vfixed(F,w) = unifier (w,T(F)[ l:|io|]) 

the most general unifier (MGU) of w and the corresponding trajectory prefix T(F) [1 : | w \ ] 
of F (where unifier(-, •) returns the most general unifier for two message patterns or 
sequences thereof, or _L if they cannot be unified). 

For a given knowledge base KB £ 2 C describing the belief state of an agent (KB is 
assumed to be encoded in the same propositional language £ as the content of messages 
for reasons of simplicity), this allows us to define 

9 P oss(F,KB,w) = = '&fixed(F,w)‘d' A 3 i.KB |= C[i]d} 

as the set of substitutions still possible under F, KB, and w. The elements of 0 poS s are 
extensions of '&fi xex [ for which at least one condition in C(F) is satisfied. 1 

For a given similarity measure a : A4* x A4* — » Ron message pattern sequences, 
we can define 



i sx ( z 7 \ i similarity frequency relevance 

^ s ^ ^ 

a(d,F)= <r{T(F)0,T(F)9{F)\i]) h 0 (F)[i ] *(F,0,KB) 

i=l 



1 We use the notation Ld here for the result of applying d to each element of a list or set L, and 
&&' for the substitution that results from applying d' after i). Further, we implicitly assume that 
only minimal substitutions are considered for •&' , only replacing variables that actually occur 
in T(F) or in some C(F) [ij. 
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to assess to which extent D is “applicable” to F. In this definition, he(F)[i\ is used to 
take the frequency of a past condition/substitution into account and Ci expresses how 
relevant a particular frame condition C[i\ is in determining the applicability of i). 1 This 
quantity can be used to derive a conditional probability distribution over the different 
substitutions d F may be enacted under: 

P(d\F,w) = 




if i? e O poss {F,KB,w) 

otherwise 



( 1 ) 



for some normalisation constant a. Finally, combining this formula with the frequency 



P(F\w) 




h T (F)[\T(F)\] 

€F,unify(T(F'-)[l:\w |],tu) h T (F' )[\T (F' )\] 



if unify (T(F)[l:\w\\,w) 
else 



of F matching any past encounter starting with w allows us to compute the continuation 
probability with which an encounter that started with w will be concluded with w': 

P(w'\w) = ^ P(d\F,w)P(F\w) 

FgF,wvj , —T(F)'S 

Looking back at figure 1, this equation defines a probability distribution for the 
possible continuations of an encounter given a current “path” in the expectation network 
induced by T . Next, we will show how this semantics of rrflnFFrA can be used to conduct 
rational reasoning about communication. 



5 Reasoning and Making Decisions About Communication 

Based on the formal model presented in the previous section, the general principles of 
InFFrA can be concretised and embedded into an agent architecture to endow agents with 
the ability to reason and make decisions about communication. To illustrate how this 
is done, we will again go through the individual steps of the abstract reasoning cycle 
depicted in figure 2. 

Interpretation and Matching/Assessment. At the beginning of each reasoning cycle, 
the knowledge base KB and the encounter prefix w are updated from the peer’s last 
utterance. 

As for matching and assessment, the agent checks if 6> poss (F a , KB , w) ^ 0, i.e. if 
the trajectory T(F a ) of the active frame F a matches w and the remaining steps of T(F a ) 
can still be executed under KB. Considerations about the desirability of F a could also 



2 A simple definition of a, considering only those conditions currently satisfied, is for example 
given by 



d{F,d,KB) = I 1 

0 otherwise. 
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play a role in the framing decision, and a possible definition for such a desirability 
criterion will be given in our description of frame enactment. 

Returning to the example of the previous section, we consider an encounter 
with T = {F cn ,F cr } and F a = F cn for both agents and prefix w = 
cfp(agent 2 , agent 1 , ( sell (tire , 1) , price Of (tire, X))), so that agent 1 can successfully 
match F a against w. If for example agent 1 did only sell tires in fours, executability of 
F a would still fail and require a re-framing. 

Adjustment/Re-Framing. The idea behind grouping different courses of interaction into 
frames is to exploit the fact that (usually) similar types of interaction (e.g. negotiation 
dialogues, contracting, etc.) exist which differ only in the specific content of messages, 
but not in what they achieve for the interacting parties. This hierarchical view enables 
agents to optimise within the current frame while disregarding other frames during a 
conversation. However, if the active frame cannot be carried out any longer, the search 
space for appropriate continuations has to be expanded. 

In this case, a variety of frame selection heuristics can be used to find suitable 
alternatives. In our implementation, experimental results for which are presented in the 
following section, we use hierarchical reinforcement learning techniques [13] to learn 
an optimal frame selection strategy over time. Also, we construct new frames through 
concatenation in a planning-like manner to achieve the original goal of a conversation 
that went awry. An extensive treatment of the resulting architecture can be found in [2]. 

At the end of an encounter - in order to maintain a concise model of past interactions 
- the active frame is augmented by a substitution that unifies its trajectory with w, along 
with a set of conditions that were required for or precipitated by the execution. 

Enactment. If the active frame contains no further steps, the agent simply terminates 
the encounter, as is the case if no active frame could be found. Else, the next message or 
action is chosen by applying the locally optimal substitution iJ‘ to the next step of the 
active frame’s trajectory. 

To determine •&*, we assume that the agent’s preference towards different world 
states is expressed by means of a real-valued utility function u : A4* x 2 C — > M, where 
u(w, KB) is the utility associated with a message sequence w being executed for initial 
knowledge base KB. 

Since some variables of 0" will be bound by the agent himself, while concrete 
values for others will be “selected” by his peer with a certain conditional probability, 
the optimal substitution is defined as the one with the highest expected utility. Normally, 
this notion of expected utility will also be used during frame assessment to determine 
the desirability of a specific substitution or entire frame. For example, one could force 
a re-framing unless executing the postfix of the active frame under the most desirable 
substitution yields a positive utility. 

If we write 0 S and 0 p for the sets of possible substitutions the agent and his peer in 
the current encounter can apply, respectively, the expected utility of executing a frame 
F under d s £ 0 S is given by 

E[u('& s \F,w,KB)]= ^2 u(postfix(T(F),w)'& s d p , KB) ■ P(d p \d s ,F,w), 
i? p 6 o p 
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where postfix (T(F),w ) is the postfix of F corresponding to prefix w (which can be 
determined by applying the most general unifier of w and the corresponding prefix of 
T(F) to the respective rest) and P(t? p |i? s , F, w) is the probability with which the peer 
will conditionally choose some substitution £ 0 p depending on the agent’s choice 

$S € F> s . 

An approximation of P('t) p |t) s , F, w) can be computed from the past cases stored in 
F. Using Bayes’ rule and applying equation 1 to both numerator and denominator, we 
can approximate 



P(t) p |t? s ,F,u>) 



^{■dfixedjF, UptMp, P) 

v(tifixed(F, F) 



if ^ fixed (F-, w)'d s ‘d p £ O poss (F , KB , w) (and 0, else). By means of standard expected 
utility maximisation, the optimal substitution i)" is then given by 



1 9*(F,w, KB) = arg max E[u(i^ s \F,w, KB)\. 
d s e&s 



Concluding the above example under the assumption that no re-framing was neces- 
sary, agent i now has to select an appropriate proposal. That is, each of the elements 
of Oposs yields a different price for a tire, and the search for the best substitution is a 
search for the highest possible price (as u will usually increase with higher profit) such 
that agent 2 will still accept (as the probability for the accept will usually decrease with 
an increasing price) and will be guided by similar past cases stored in 0(F cn ), i.e. by 
past prices for tires. 



6 Experimental Results 

In order to show their performance in practice, the concepts presented so far have been 
implemented and tested in the multiagent-based link exchange system LIE9CN [9]. In 
this system, agents representing Web sites engage in communication to negotiate over 
mutual linkage with the end of increasing the popularity of one’s own site and that of 
other preferred sites. 

Available physical actions in this domain are the addition and deletion of numerically 
rated links originating from one’s own site and the modification of ratings (where the 
probability of attracting more traffic through a link depends on the rating value). 

LIE9CN provides a highly dynamic and complex interaction testbed for the following 
reasons: 

- Agents only have a partial and incomplete view of the link network. In particular, 
agents engage in non-communicative goal-oriented action in between encounters, 
so that the link network (and hence the agents’ utility situation) may change while 
a conversation is unfolding. 

- The number of possible link configurations is vast, and agents can only predict 
possible utilities for a very limited number of hypothetical future layouts. 

- There is no notion of commitment - agents choose frames in a self-interested way 
and may or may not execute the physical actions that result from them. Also, they 
may undo their effects later on. 
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LIE 9 CN agents consist of a non-social BDI [ 8 ] reasoning kernel that projects future 
link network configurations and prioritises goals according to utility considerations. 
If these goals involve actions that have to be executed by other agents, the m InFFrA 
component starts a framing process which runs until the specific goal has been achieved 
or no adequate frame can be found. We report on experiments in which agents were 
equipped with frames with the following six trajectories: 

request(A, B, X) — > accept(E, A, X) — » conf irm(A, B, X) — > do (B,X) 
request(A, B , X) — > propose(S, A, Y) — > accept(A, B , Y) — ■> do (B, Y) 
request(A, B, X) — > propose — also(S, A, Y) — > accept(A, B , Y) — * 
do (B,X) -> do(A,Y) 
request(A, B , X) — * reject (B, A 1 X) 
request(A, B , X) — > propose(E, A, Y) — > reject (B, A, Y) 
request(A, B , X) — > propose — also(B, A, Y) — > reject(f?, A, X) 

The first three frames allow for accepting to perform a requested action X, making 
a counter-proposal in which Y is suggested instead of X, or using propose — also 
to suggest that B executes X if A agrees to execute Y. The last three frames can be 
used to explicitly reject a request or proposal. In that, X and Y are link modification 
actions; each message is available in every state and incurs a cost that is almost negligible 
compared to the utilities gained or lost through linkage actions (yet high enough to ensure 
no conversation goes on forever). Also, agents can always send a stop action to indicate 
that they terminate an encounter if they cannot find a suitable frame. 

After their termination, encounters are stored in the frame from which 
they have originated. For example, agent U\ would store the encounter 
requester, 02, add(a2, or, 2 )) — > reject(ai, 02, add(a2, or, 2 )) by adding a sub- 
stitution \A/a\, B/d2, X/add(a2, or, 2 )] to the respective frame together with an auto- 
matically generated list of conditions that were required for physical action execution. 

As state abstraction, we use generalised lists of statements of the form 
{T|I}({^j -R}, {/, R, T}, {+, — , ?}) representing the physical actions talked about in 
an encounter, j and | stand for a positive or negative link modification (i.e. addi- 
tion/deletion of a link or an increase/decrease of its rating value), HR for the initiator 
and responder of the encounter, T for a third party; +/—B indicates whether the (learn- 
ing) agent likes, dislikes or doesn’t know the target site of the link modification. For 
example, if ai and 02 talk about do(oi, deleteLink(ai,az)) in an encounter initiated 
by ai (while the learning agent 02 is the responder and likes a3’s site) this is abstracted 
to J,(J, T, +). If in the same conversation 02 suggests to modify his own link toward 
a\ (whom he does not like) from a rating value of 1 to 3 , the state (viz subject ) of 
the encounter becomes { J. (I, T, +) , j (R, /,—)}. The intuition behind this state abstrac- 
tion method is to capture, in a generalised form, the goal of the conversation that can 
currently be realised while at the same time reducing the state space to a reasonable 
size. 

Figure 3 shows a comparison for a system with ten agents with an identical profile 
of private ratings (preferences) towards other agents (both plots show the performance 
of the best and the worst agent in the group as well as the average utility over all agents). 
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Fig. 3. Performance plots for communicating BDI agents and m InFFrA agents 
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In the first plot, agents employ BDI reasoning and additionally send requests to others 
whenever they favour execution of someone else’s action according to their BDI queue. 
These requests are then enqueued by the recipient as if he had “thought of” executing 
the respective action himself. Thus, it depends on the recipient’s goal queue and on his 
utility considerations whether the request will be honoured or not. As one can see, after 
a certain amount of time agents do no longer execute any of the actions requested by 
others, and cannot find any profitable action to execute themselves, either. The system 
converges to a stable state. 

The second plot shows the results of a simulation with the same setup as above but 
using m InFFrA agents. Again, agents issue requests whenever they identify that someone 
else could do something useful. After this initial message, the framing procedure takes 
over. Quite clearly, despite the fact that there is a greater variation in maximal, minimal 
and average agent utility, the average and the best agent perform significantly better than 
in the BDI case, while the weakest agent performs just as good as in the BDI case on 
the average. 

While on first glance the performance of m' InFFrA agents might not be strikingly 
different from those using plain BDI, it should be noted that the results establish a 
lower bound on the performance gained by using nflnFFrA. In environments with (pos- 
sibly) non-benevolent peer agents showing non-stationary behaviour, the performance 
of agents using prespecified communication protocols or assuming a fixed semantics of 
communication can become arbitrarily bad, while m InFFrA includes the ability to learn 
what to expect from a given peer in a specific interaction situation. To allow for any 
quantitative statements, however, additional experiments will be necessary. 

Yet another interesting interpretation can be drawn from the fact that the average 
utility of m InFFrA agents lies within the range of the two horizontal lines in the plot. 
These denote the average utilities for two special linkage configurations: the lower one of 
the two corresponds to a fully connected linkage graph, in which each agent (honestly) 
displays the ratings of his out-links (i.e. reveals his true opinions about others). The 
slightly higher utility shown by the upper line is attained if agents do not lay any links 
toward agents they dislike. It is an interesting property of the utility function used in 
LIE9CN. that acting “politically correct” is slightly better than being honest. The fact 
that agent utilities evolve around these benchmarks indicates that they truly strive to 
make strategic communication moves and exploit the advantages of concealing certain 
beliefs. 

Apart from these utility results, two interesting observations can be made about the 
framing behaviour: 

- Agents never use the reject-frames, because they immediately notice that halting the 
encounter right away is cheaper (by the cost of sending the re j ect message) and has 
the same effects. This allows the frame designer to focus on “success” trajectories 
without having to worry about failure. 

- Once agents accept to do something, they will keep their promise. This is simply 
because they are able to predict the utility consequences of an encounter and have 
no reason to engage in communication if they do not intend to proceed. 
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7 Conclusions 

In this paper we have presented an architecture for reasoning about communication 
patterns within the framework of empirical semantics. From a practical point of view, 
this contributes to the “agent side” of ACL research, as it offers methods to construct 
and implement agents that are able to deal with given specifications of rules that govern 
the communication in a system. Three features are particularly interesting about our 
approach: 

1. The rri InFFrA frames used in our formal model combine information about the 
surface structure of communication sequences, logical constraints (frame condi- 
tions) and empirical data (counters). This allows for great flexibility with respect 
to what is defined in commonly agreed protocols, semantics, etc. For example, in 
non-exploitable protocols as those used in mechanism design, it may suffice to just 
specify admissible message sequences (because it does not matter what the agents 
think, for example), while other types of interactions such as contracting may require 
commitment rules, descriptions of agents’ mental states, etc. 

2. Agents are capable of exploiting past communication experience. Thus, they are 
able to “start out” with a predefined set of patterns and to test to which degree 
their peers comply with them, which offers a major advantage in open systems 
where adherence to protocols, truthfulness constraints, etc. cannot be guaranteed. 
In particular, this paves the way for the use of machine learning methods (as we 
have done) that make minimal assumptions about others’ behaviour and simply 
accumulate communication knowledge as they go. 

3 . The architecture combines decision- theoretic (probabilistic) reasoning methods with 
symbolic communication. This constitutes an important contribution to the practi- 
cal design of communicating agents, but also an initial step to explore the special 
character of communication as opposed to general action. In [10], for example, we 
have argued for trading off optimality against predictability in communication so 
that more reliable communication patterns can occur. Using decision theory as a 
firm foundation for rational reasoning can help formalising what agents (should) do 
in communication and whether and in which way this differs from the models used 
in, e.g. game theory and economics. 

A major advantage of our approach is that it allows us to combine the decision- 
theoretic power of RL models with the knowledge-based aspects of symbolic agent 
communication, interaction protocols and ACL research in general. It is this aspect that 
makes rational action and learning possible for high-level agent architectures that employ 
logical reasoning. 

In the future, we intend to look at more complex models of interaction frames with 
trajectories more expressive than simple sequences. Also, we want to investigate to 
which degree m InFFrA agents are capable of exchanging meta-frame information to 
reach consensus about which frames to use. Finally, a lot of work needs to be done 
on automatically transforming ACL and interaction protocol specifications into readily 
usable rrflnFFrA frames. 
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Abstract. Previous work by other researchers has investigated the evo- 
lution and stability of a ‘probabilistic reciprocity’ strategy in a package 
delivery domain where all agents could communicate reputations to ev- 
ery other agent. We extend that work to the more realistic situation 
of spatially distributed agents with neighborhoods that restrict agent 
interaction and communication. We improve the original probabilistic 
reciprocity strategy with a modification that tarnishes the reputations 
of agents that repeatedly refuse requests for help. We then investigate the 
effect of reducing neighborhood size from the general case of the ‘global 
neighborhood’ used in previous work. Our experiments show that neigh- 
borhoods can be reduced to a critical size without a significant degrada- 
tion in the evolutionary stability of the improved probabilistic reciprocity 
strategy. We also show that locating like agents within a niche can miti- 
gate this degradation. From a multi-agent design perspective, this means 
that for a population with a given proportion of selfish and dishonest 
agents, communication may be reduced to within a subset of the popu- 
lation while retaining the same success of the reciprocative strategy. We 
also show how to extend the problem domain to abstract a wider range 
of interaction situations as defined in the literature. 



1 Introduction and Motivation 

The importance of the field of Multi- Agent System (MAS) research is well rec- 
ognized and the emergence of workshops and conferences such as Cooperative 
Information Agents (CIA) and Cooperative Information Systems (CoopIS) ac- 
knowledges cooperation as a critical area of investigation within the field. 

Cooperation and interaction are common to the very definitions of agency 
in the literature. Among Ferber’s [1] minimal common properties of an agent 
is the ability to communicate directly with other agents. Maes [2] talks about 
autonomous agents being “engaged in a cooperative process” with a user. In 
Wooldridge and Jennings’ weak notion of agency [3], an agent has “social abil- 
ity”. According to Weiss [4], the key question for MAS research is “When and 
how should which agents interact — cooperate and compete — to successfully 
meet their design objectives?” . He mentions abilities such as improving the state 
of coordination, reconciling disparate viewpoints and conflicts, negotiating and 
forming organizational structures. Clearly, interaction and specifically coopera- 
tion are very important to the design of a multi-agent system. 



M. Klusch et al. (Eds.): CIA 2004, LNAI 3191, pp. 123-136, 2004. 
(c) Springer- Verlag Berlin Heidelberg 2004 
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Popular MAS frameworks such as JADE [5] reduce the agent environment 
to a passive message transport system. The advantages to a MAS of a rich and 
active environment beyond a message transport system are evidenced by suc- 
cess with experiments on algorithms inspired by social insects [6]. These have 
demonstrated that virtual gradient fields and chemical trails can aid agent com- 
munication and coordination [7]. 

Research toward a better understanding of such agent-environment interplay 
is in its infancy. One worthwhile endeavor is investigating under which envi- 
ronmental conditions there are benefits to the agent society and under which 
conditions the imposition of an environment may be detrimental to the desired 
societal level behavior. 

In particular, we are investigating the effect of introducing spatial distribution 
and neighborhoods to Sen and Dutta’s work [8] on the evolution and stability of 
agent strategies employing a probabilistic reciprocity decision rule [9] . A spatially 
distributed environment could arise where either agents are actually physically 
dispersed or where agents represent spatially dispersed users. 

Ubiquitous computing may be an example of the former case. Imagine a 
shopping mall where some tenant shops have situated agents connected to a 
wireless network. One possible commercial direction for agent research is toward 
the idea of these agents sharing information on potential customers and their 
requirements and tailoring services to those requirements. Agents representing 
different categories of outlet should like to establish a reciprocal relationship for 
identifying potential customers in their vicinity and directing those customers 
to one another. Could such a relationship develop and be sustained? 

Secondly, cases will arise where agents are pooled in a software environment 
but represent physically dispersed users. This is true in the VLAB system [10] 
which offers a brokerage service to hauliers represented by agents. The mod- 
ern haulage industry is pan-continental and offers cooperation opportunities for 
sharing haulage resources such as idle tractor cabs and empty trailer space. 
However, over large geographical distances and for infrequently visited delivery 
locations, there would be fewer interaction occasions for hauliers. Under what 
conditions would parasitic agent strategies dominate such a system? 

This paper has the following outline. We first cover the previous related work 
on cooperation and probabilistic reciprocity. Section 3 describes our experiment 
setup and how the problem domain we use could be modified to fit within many 
recognized categories of interaction situation. Section 4 details the experiments 
we have run and their results. Section 5 draws conclusions from these results 
and suggests some directions for extending our work. 



2 Related Work 

2.1 Deterministic Strategies 

Among the most frequently cited work on cooperation is that of Axelrod [11, 12, 
13]. This work investigated the robustness of various deterministic strategies in 
playing the Prisoner’s Dilemma [14]. It has been criticized on several fronts. 
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Game theorists and economists have pointed out weaknesses in Axelrod’s 
conclusions about the merits of the TIT-FOR-TAT strategy [15]. They highlight 
his choice of initial conditions as biasing the basin of attraction into which the 
game’s dynamics settle. 

The applicability of Axelrod’s conclusions to real-world problems has 
been criticized by Sen and Dutta [8] . They highlight weaknesses in the following 
areas: 

— Initial Decision: Axelrod assumed TIT-FOR-TAT agents started by cooper- 
ating. This leads to all agents cooperating subsequently. However, if agents 
start by not cooperating then TIT-FOR.-TAT will never produce cooperative 
interactions. 

— Symmetrical Interactions: Axelrod assumed every interaction is perfectly 
symmetrical. That is, in any interaction, both agents incur the same absolute 
value of cost and benefit. In real life these values are not always the same. 
In deciding whether or not to help others, an agent must consider both the 
history of past interactions and expectation of interactions in the future. 

— Repetition of identical scenarios: in real life, either the parties involved or 
the environmental conditions change between interactions. 

— Lack of a measure of work: to compare different scenarios, we need some 
common metric of the work involved. A simplistic TIT-FOR.-TAT strategy 
that does not incorporate such as measure would favor equally taking on a 
very expensive task and a relatively cheap task. 

Other researchers have achieved improved results with deterministic variants 
of the TIT-FOR.-TAT strategy such as PAVLOV [16]. However, all of these exper- 
iments are set in the highly abstract Prisoner’s Dilemma game. This abstraction 
does not capture the following features that are very real considerations in a rich 
multiagent society. 

— The possibility of communication: This precludes any reasoning with another 
player, threats and bargaining. 

— The influence of third parties: The game is only between two players. In 
multi-person interactions, a witness to a defection can punish the cheater. 
A victim of a cheater can warn others of its experience. 

— The problems of implementing a choice: All strategies are perfectly imple- 
mented. There is no mention of any domain specifics and no possibility of 
partial cooperation and defection. 

— Uncertainty about what the other player actually did on the previous move: 
Strategies as they are perceived are assumed to represent the player’s inten- 
tion to cooperate or defect. The possibility of noise and errors in perception 
is not addressed. 

2.2 Probabilistic Strategies 

Sen has proposed a probabilistic reciprocity strategy that addresses some of 
the weaknesses in Axelrod’s work [9] . This strategy uses a sigmoidal probability 
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function of additional cost of helping, average cost of tasks, and reputation of the 
requesting agent to decide which requests for help to honor. We briefly repeat 
its definition here for continuity. 

Let there be N agents in the multiagent system, each with T tasks assigned 
to it. The jth task assigned to the fth agent is tij. Let Ski be the cumulative 
savings obtained by agent k from agent i over all of their previous interactions. 
Let Wki be the cumulative work done by agent k for agent i over all of their 
previous interactions. Then the balance of these exchanges is Bki = Ski — Wki- 
The balance of their exchanges between two agents that have never interacted 
before is zero. Balances are not symmetrical Bki 7^ Sen suggests using (1) 
to calculate the probability that an agent k, with an assigned task tki, will carry 
out an additional task t t j for a requesting agent i. 



Pr (, k,l,i,j ) 



1 + exp 



1 

C*-/3xC* vg -B ki 



(1) 



An agent’s willingness to cooperate is measured by (3 and r determines the 
steepness of the cooperation curve. C„ vg is the average cost of tasks to be per- 
formed by agent k. CA is the extra cost agent k incurs by accepting agent i’s 
task j. 

The parameters (3 and r are global constants. The values of the other vari- 
ables come from the dynamics of the system, specifically the agent’s history of 
interactions and the tasks it is randomly assigned. As the exponential term in- 
creases, the probability of helping the requesting agent decreases. Therefore, a 
negative balance with the requesting agent can only decrease the probability of 
helping that agent. 

The advantages of a probabilistic approach over a deterministic approach are 
that it can bootstrap cooperative relationships despite the potential for cheating 
and it can better handle noise. Furthermore, Sen’s proposed strategy favors 
helpful agents over selfish agents since it acknowledges the history of interactions 
with a requesting agent and it allows for current workload. 



3 Experimental Setup 

In these experiments, we aim to extend the work of Sen and Dutta to a spa- 
tially distributed agent environment. Our implementation is based on our un- 
derstanding of their work as reported in [8] . We make clear where we have made 
assumptions about their implementation. 

3.1 Agent Problem Domain 

The package delivery problem domain consists of one depot to which three radial 
fins are attached (Figure 1). All fins have a length of three units. A package 
delivery task involves bringing a package from the depot to a position on a fin. 
There is a single depot that is used by all agents. 
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Fig. 1 . Schematic of the Package Delivery Domain showing a central depot with three 
radial fins all of length 3 

We retain Sen’s metaphor of the package delivery domain but stress that we 
do not intend it to be a model of real-world physical delivery locations in the 
way, for instance that directed graphs might express vehicle routing. Rather, 
we consider it an abstraction of types of cooperative interaction. We propose 
that the specific interaction type, as defined by Ferber’s taxonomy of interaction 
situations [1], can be set by some adjustments to the domain as outlined below. 

— The situation of incompatible goals is modeled by setting a limit on the 
number of packages that can be held at a given delivery location. When 1 
below that limit is reached, then one agent’s delivery to that location will 
prevent another agent’s subsequent delivery to that location. Those agents’ 
goals are thus incompatible. 

— Conflict over resources can be introduced by limiting the number of 
agents on a fin at the same time. 

— Agent capacities can be modeled by giving each agent a finite amount of 
fuel. Agents then use fuel in proportion to the distance of a task from the 
depot. If an agent has insufficient fuel to reach a delivery location it has 
insufficient resources for its task. 

— Agents cannot cross between fins for different concurrent tasks. This repre- 
sents tasks for which one agent cannot facilitate another by taking on its 
task. Increasing the number of fins therefore reduces cooperation oppor- 
tunities. 

— Agents can only interact at the depot. For task sets evenly distributed over 
fins and fin positions, increasing the fin length reduces the frequency of 
interaction and increases the conflict over resources. 

The problem domain as reproduced from Sen and Dutta’s work therefore 
models compatible goals with sufficient skills and sufficient resources. This fits 
within Ferber’s ‘indifference’ category. 

However, what makes this problem domain more interesting is that agents 
desire to minimize their costs when achieving their tasks. There is an incentive 
for agents to facilitate one another by sharing tasks. Costs are calculated as 
follows. 




128 E. Ridge et al. 



Consider a package delivery task at a distance d from the depot. The work to 
complete this task involves traveling a distance d from the depot to the delivery 
location on the fin and returning back to the depot for the next task. It costs one 
unit per unit distance traveled per package and one unit per distance traveled 
(empty-handed) on the return to the depot. The cost of this work for such a task 
in isolation is d + d. However, if two tasks are undertaken at the same time then 
the savings on the single return journey mitigates the extra cost of bringing the 
additional package to its destination. 

For example, let d\ be the maximum distance of all tasks the helping agent 
k is planning to undertake. Let d 2 be the distance of the additional task j that 
the agent is considering undertaking for agent i. Then (2) gives the extra cost 
of the additional task to agent k. 



(dr, da) 



d 2 if d 2 < di 

d 2 + (d 2 — d\) otherwise 



(2) 



The implication of this cost structure is that work performed cooperatively 
costs less than the same work performed in isolation. 

Each agent is given a randomly generated list of assigned tasks such that every 
agent receives the same total number of tasks and the same total amount of work. 
Agents then proceed to complete their tasks either individually or cooperatively 
based on the result of their interactions at the depot. There is no limit on the 
number of additional tasks that an agent can take on simultaneously. 



3.2 Agent Environment 

For the concept of a neighborhood to have meaning we require some form of 
spatial distribution. We have used the common setup of a square lattice of con- 
nected nodes where an agent is located at a node [17]. The top and sides of the 
lattice are wrapped to avoid possible edge effects. 

The lattice that agents inhabit has no relation to the abstract package deliv- 
ery problem domain. Conceivably, agents on the lattice could cooperate accord- 
ing to some other abstraction such as the Prisoner’s Dilemma game. 

In this lattice framework, we can define several types of neighborhood. Two of 
the most common variants are the Moore neighborhood and the Von Neumann 
neighborhood as illustrated in Figure 2 for a neighborhood range of two. 

We have chosen the Moore neighborhood because by increasing its range suf- 
ficiently we can recover the general case of global environment vision used in 
Sen and Dutta’s work. Neighborhoods introduce restrictions on both agent in- 
teractions and on the evolutionary selection scheme. Neighborhoods restrict an 
agent’s interactions at the depot to only those agents within its own neighbor- 
hood. Neighborhoods restrict the sampling of the evolutionary selection scheme 
from the whole population to that subset within an agent’s neighborhood. We 
use this restriction because the evolutionary scheme is an attempt to model an 
agent’s adoption of new strategies based on its perception of successful strategies. 
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Fig. 2. Moore neighborhood (left) and Von Neumann neighborhood (right) 



3.3 Evolutionary Selection 

The evolutionary selection scheme is not intended as a model of any real evolu- 
tionary process. Rather it introduces a selection pressure on agent strategies. It 
is a simple scheme that we reproduce from the previous work we extend. 

At the end of an evaluation period, when each agent has completed all tasks 
it was assigned at the start of the period, each agent’s cost C incurred during 
the period is totaled. An agent’s fitness / is given by: 



/ 



1 

1 + C 



( 3 ) 



We use the 1 term in the denominator to avoid division by 0 in the unlikely 
event of an agent incurring 0 cost during the whole evaluation period. 

Two different agents from an agent’s neighborhood are selected with prob- 
abilities proportional to their fitnesses. The strategy of the selected agent with 
the greater fitness is adopted for the next evaluation period. 



3.4 Agent Help Decision Strategies 

The most sophisticated strategies used by Sen and Dutta were what they termed 
Earned Trust Reciprocative (ETR) and Collaborative Lying (CL). 

Earned Trust Reciprocative Strategy. The ETR strategy uses the proba- 
bilistic reciprocity function of Equation 1 in making its help decisions. The B ^ 
term is the balance reported by all agents with whom the deciding agent has 
an existing balance greater than zero. That is, all agents that have helped the 
deciding agent in the past and are therefore presumed honest. 

Collaborative Lying Strategy. The CL strategy never helps other agents. 
In addition, when asked for a reputation it bolsters the reputation of unhelpful 
agents and tarnishes the reputation of helpful agents. No claim is made for the 
rationality of CL. It exists purely as a disruptive strategy. 

Differences in Implementation. There may be some differences in our im- 
plementation of these strategies due to the following details. 
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— Average Cost C,j vg : There are several ways to calculate the average cost term 
in the probabilistic reciprocity equation. We have chosen a preset value of 
2.5. This is an approximate average over the possible combinations of one 
assigned and one additional task in a domain with three radial fins all of 
length three. 

— Requests for Help: There are many ways to assess requests for help. We have 
chosen to assess each request in a bundle independently of others in the bun- 
dle. This means that additional cost is calculated relative only to the current 
assigned task, regardless of other additional costs that may arise after as- 
sessing the bundle and finalizing the transfer of tasks with requesting agents. 

— Updating Balances: When updating balances after an interaction has com- 
pleted we use the additional cost that was calculated when assessing requests. 
While this may no longer be the true additional cost, it is nonetheless the 
value on which the decision to help was made and we are evaluating the 
strategies that use this decision process. 

We now propose an additional strategy that is a variant of Sen’s ETR Strat- 
egy- 

Earned Trust Tarnish Strategy. The Earned Trust Tarnish (ETT) strategy 
differs from ETR in one way. When an agent refuses an ETT agent’s request 
for help, the ETT agent decreases its balance with the refusing agent by a 
constant. This modification attempts to address a potential weakness in the 
ETR strategy. An ETR agent that repeatedly requests help from a possibly 
selfish CL agent keeps no record of its lack of success in receiving help from 
that agent. Yet should that CL agent request help from the same ETR agent it 
will still be treated as a possibly cooperative agent. This is unrealistic when the 
CL agent has no history of honoring requests for help. By reducing its balance 
with refusing agents, the ETT strategy helps identify likely selfish agents and 
will treat requests from those agents with greater skepticism than an ETR agent 
would. Figure 3 illustrates this for an average cost of 2.5 and a decreasing range 
of balances (0,-1,— 2). 

The topmost curve plots an ETT agent’s probability of helping against in- 
creasing additional cost when balance with the requesting agent is 0. This is the 
same probability that is always used by an ETR agent. The other curves show 
decreasing probabilities of helping for corresponding decreases in balance with 
the requesting agent. 

The tarnishing constant must be set so that the likelihood of help is reduced 
without completely ruling out help. This acknowledges that the refusals may have 
been due to the requested agent being overloaded during previous interactions. 



4 Experiments and Results 

This section outlines the experimental setup and results obtained. Since we aim 
to extend Sen and Dutta’s work, we set all parameters to be the same as theirs. 
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Extra Cost of Additional Task 



Fig. 3. The effect of decreasing balance on Probability of helping for an ETT agent 

Specifically, there are 3 fins all of length R = 3. There are N = 100 agents. 
Willingness to cooperate is set to /3 = 2.0 and r is set to 0.75. We repeat each 
experiment three times. In addition, we introduce the following parameters and 
details common across all experiments. The square lattice contains 10 x 10 nodes. 
Agents are dispersed randomly on the lattice and remain static for the duration 
of the experiment. 

4.1 Reproducing Previous Work 

The initial percentage of Collaborative Liars was controlled from 10% to 90% 
in increments of 10 with the remaining agent strategies initially set to Earned 
Trust Reciprocative. Every agent was given a neighborhood range sufficient to 
see the whole lattice. On the 10 x 10 lattice with Moore neighborhoods this value 
is 5. Figure 4 summarizes the results 1 . 

Qualitatively, our results show the same trend as Sen and Dutta’s with ETR 
requiring a roughly constant minimum average amount of tasks to dominate 
the population. The disparity in the quantity of tasks and the sharp increase 
at high initial percentages of CL must be due to some of the differences in our 
implementation as noted previously. However, the qualitatively similar trend 
make us confident that we have reproduced the most important characteristics 
of the agent interactions. Henceforth, we use our implementation of ETR as our 
base point. 

4.2 Tarnishing Repeated Refusers 

We now repeat the previous set of experiments, replacing the ETR strategy 
with the proposed ETT strategy. The tarnishing constant for the ETT strategy 
is set to 1. Figure 5 shows that the ETT strategy requires less than half the 
mean tasks of ETR before it dominates the CL strategy, a marked performance 
improvement. 



The Sen and Dutta curve is estimated from their paper. 
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Fig. 4. Average of Minimum Tasks before ETR agents dominate CL agents 




Initial % of Collaborative Liars 



Fig. 5. Average of Minimum Tasks before ETT agents dominate CL agents 



ETT identifies potentially selfish agents more quickly than ETR and so less 
time is wasted helping those agents. The ETT strategy suffers from the same 
sharp degradation in its performance as ETR at high initial percentages of CL. 
This is illustrated in Figure 6 which shows the percentage increase in mean tasks 
required by both ETR and ETT as the initial percentage of CL is increased. 

Having established that ETT is a better strategy in this domain, we proceed 
to investigate the effect of neighborhoods on that strategy. 

4.3 Introducing Neighborhood Effects 

In this set of experiments, we repeat the previous test of ETT against CL for 
the set of all possible neighborhood ranges. In a 10 x 10 lattice with Moore 
neighborhoods this set is {4, 3, 2, 1}. These ranges correspond to proportions of 
the environment {80%, 50%, 25%, 10%}. Figure 7 summarizes the results. 

Up to about 30% initial CL, there is little difference between the different 
neighborhood ranges apart from the extreme neighborhood range corresponding 
to 10% of the environment. Beyond this point, however, the ETT behavior starts 
its steep increase. The lower neighborhood ranges begin to increase first. This 
is because, with a random distribution of strategies on the lattice, the proba- 
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Fig. 6. Percentage increase in mean tasks for ETR and ETT 
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Fig. 7. Average of Minimum Tasks before ETT agents dominate CL agents for all 
neighborhood ranges 



bility of an ETT agent being located in a very selfish neighborhood is higher 
for smaller neighborhoods. Each neighborhood range has a critical point above 
which the ETT strategy cannot cope with increases in the initial percentage of 
CL strategies in the environment. Alternatively, looked at from a more positive 
perspective, there is a lower bound on neighborhood range to which commu- 
nication can be restricted without compromising the cooperation opportunities 
available at an agent vision permitting global communication. 

It should be noted that although all of the strategies have a neighborhood 
restriction imposed on them, none of them explicitly attempt to incorporate 
neighborhood effects into their calculations. Clearly, the presence of neighbor- 
hoods has a detrimental effect beyond a critical concentration of CL initially in 
the population when strategies are dispersed randomly. 

4.4 Niches 

We now test whether biases in agent location can mitigate the detrimental ef- 
fects of reduced neighborhood ranges. In this set of experiments, agents are not 
dispersed randomly on the lattice. We initiate the experiments with the ETR 
strategy in a square niche of various sizes. 



134 E. Ridge et al. 



By niche, we mean a group of adjacent agents using the same strategy. This 
is similar to what Axelrod terms territoriality [12]. We describe the size of a 
square shaped niche by its range. For a square shaped niche with l nodes on 
each side, we define the niche range r as: 



l-l 



Figure 8 illustrates a 6 x 6 niche (range 2.5). 



(4) 




We emphasize that neighborhoods arise from individual agents having lim- 
ited vision of the world whereas niches are a biasing in the dispersion of agent 
strategies on the lattice. Figure 9 summarizes the results for a 6 x 6 niche. 

The solid plot is for a 6 x 6 niche of 36 ETR agents surrounded by CL. The 
dashed plot is for 36 ETR agents randomly dispersed on the lattice. For all neigh- 
borhood ranges, the ETR agents that begin in a niche outperform those that 
are randomly dispersed. This effect is increased when the neighborhood range 
of agents approaches and decreases below the range of the niche, represented by 
the dashed vertical line. Collaborative agents with neighborhoods smaller than 
their niche are no longer exposed to unhelpful agents outside the niche. This 
shows that if we wish to reduce the number of tasks for which an ETR strategy 
can dominate a population, we should locate that behavior in a niche, preferably 
larger than the neighborhood size available to agents. 

5 Conclusions and Future Work 

We have validated aspects of previous work in the field and generalized those 
experiments to an environment with a spatial distribution and neighborhoods. 
We have improved on the proposed probabilistic reciprocity strategy for the 
given problem domain by modifying the strategy to tarnish repeated refusers. 

Taking this improved reciprocative strategy, we have investigated the effect 
of neighborhoods on the performance of the strategy where neighborhoods re- 
strict agent interactions. Our results show that above a threshold initial selfish 
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Niche Range 2.5 (6x6 Niche) 




Fig. 9. The effect of a niche on the performance of ETR 



fraction in a randomly distributed global population, restricting interaction to a 
subset of the population does not have a detrimental effect on the performance 
of the strategy in terms of mean tasks required to dominate the population. Fur- 
thermore, when a strategy is located in a niche, it performs better than when 
randomly dispersed. This benefit is increased when the range of the niche is 
greater than the range of the agents in the niche. This is important both where 
communication has a real cost and where designers would like to reduce the 
messaging load on system agents such as Directory Facilitators without compro- 
mising the incentive to cooperate. 

We have highlighted the merits of the package delivery domain as an ab- 
straction that can be customized to model various types of interaction within 
Ferber’s taxonomy [1] beyond the ‘indifference’ category used by Sen et al. 

There are several directions in which to take future work. There remains 
plenty of scope for variants of the probabilistic reciprocity strategy. In the context 
of spatial environments, we intend to investigate the impact of niches on the 
success of strategies and whether some strategies might be better suited to niches. 
We wish to investigate the relationship between niche size and agent vision in 
the context of the evolution and stability of reciprocative strategies. 

As already mentioned, the problem domain provides the opportunity for 
studying several categories of interaction situation and how well the existing 
strategies cope with these different interaction situations. 

Finally, we would like to look at evolutionary techniques, perhaps in combi- 
nation with learning, that could help agents improve their own values of /3, r etc 
rather than having the designer pre-set them. 
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Abstract. Many researchers have focused their efforts on developing collabora- 
tive recommender systems. It has been proved that the use of collaboration in such 
systems improves performance, but what is not known is how this collaboration 
is done and what is more important, how it has to be done in order to optimise the 
information exchange. The collaborative relationships in recommender systems 
can be represented as a social network. In this paper we propose several measures 
to analyse collaboration based on social network analysis. Once these measures 
are explained, we use them to evaluate a concrete example of collaboration in a 
real recommender system. 

Keywords: Recommender Systems, Collaboration Analysis, Electronic Commu- 
nities, Social Networks, Trust 



1 Introduction 

In the real world, not only society in general but in particular our friends, help us to 
discover new things which they think we would like. Our friends advise us about an 
interesting product, a movie, a book or a restaurant, collaborating with us in the selection 
process. Being aware of this collaboration in the real world, researchers have focused on 
the development of recommender systems [14] which can recommend items to a user 
based on information from other users. 

Particularly, the collaborative filtering method has proved to be a useful method to 
take advantage of the collaborative world especially when combined with other tech- 
nologies in a hybrid approach [2, 6]. Thus, the collaboration among users increases the 
performance of recommender systems. However, we do not know many things about 
how this collaboration is done. This is a first step towards the design of new methods and 
techniques that will contribute to optimise collaboration with a given purpose (goal). 

Recently, collaboration has been modeled as a network of users exchanging informa- 
tion, that is, a social network. Users are represented as actors (nodes) and collaborative 
relationships as directed ties. In this paper we use this representation to propose several 
measures based on social network analysis in order to understand how users collaborate. 

To illustrate the use of the measures, we perform the evaluation of a real collaboration 
framework implemented in our group. 



M. Klusch et at. (Eds.): CIA 2004, LNCS 3191, pp. 137-151, 2004. 
(c) Springer- Verlag Berlin Heidelberg 2004 
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Thus, this work is a first step to achieving a further goal. The long-term aim of our 
work is to find out how can we tune the different parameters of our recommender system 
in order to have a social network featured in an optimal way according to certain criteria. 

This paper is structured as follows. Section 2 introduces social networks and why they 
are used in our work. Our proposal of measures to analyse collaborative recommender 
systems are presented in Section 3. Section 4 introduces the collaborative recommender 
system implemented in our group and used as a basis for our experimentation and 
Section 5 shows how the proposed measures are used to analyse our real example. 
Section 6 presents related work and finally, some conclusions and further work are 
provided in Section 7. 



2 Social Networks 

A social network [3] is a representation of the relationships existing within a community. 
Social networks provide us with a tool to study collaboration, in particular through theory 
developed in social network analysis [18, 16, 7]. 

Even within the same community several types of social networks can be built de- 
pending on the social relationship taken into account: friendship, mutual support, co- 
operation and similarity are typical criteria used in establishing the social relationship 
components of a community. Actors in this social network can be individuals, groups of 
people, objects or events as far as certain relationships hold them together. The strength 
of a tie may range from weak to strong depending on the quantity, quality and frequency 
of the exchanges between actors [12]. 

In this way, social networks are able to represent societies and relationships among 
individuals from these societies by means of a graph. In collaborative recommender 
systems, each system user is represented by an actor in the graph, and relationships 
among these users are represented through directed ties. If user A develops a relationship 
with user B, there should be a directed tie from A to B. 

Since relations among users change over time, it is important to take into account 
that social networks are dynamic. So, a social network represents relationships among 
users at a certain moment in time. 

It is also important to know which locality the analysed system has. A system with 
locality 1 is the one where only the immediate ties a user has are taken into account. 
In systems where this locality is higher than 1, immediate and also indirect ties are 
considered. For example, if there is a user A connected to another user B, and B is 
connected to a third user C, A can reach C through B. However, in systems with locality 
1 , A cannot reach C unless there is a tie between them. 

3 Social Network Measures 

This section presents the measures we propose in order to analyse the collaboration 
among the users/agents of a recommender system. In general, the social network result- 
ing from a collaborative recommender system has locality 1 . Therefore, the measures 
proposed in this paper only take into account the immediate ties among the actors. 




Collaboration Analysis in Recommender Systems Using Social Networks 139 

All these measures are based on social network analysis; namely size, density, degree 
centrality, network centrality, clique membership and factions . 



3.1 Size 

Size is the number of actors present in the network, and is useful in order to calculate 
other measures. 

This parameter can give us a general idea of how the network is. Say we have a 
small firm with only 10 workers. It would be easy for each worker to know the others 
and build up relationships. Now imagine we have a firm with 1000 workers. It would 
be extremely difficult for any worker to know all of the others. As a group gets bigger 
(and size increases) the proportion of ties that could be present decreases, and usually 
partitioned groups emerge. 

3.2 Density 

Fully saturated networks (i.e. one where all logically possible ties are actually present) 
are rare, especially in social networks with a considerable number of actors. In a network 
whose size is K, the number of possible different directed ties is (K * (K-l)). 

Density is the proportion of all ties that could be present that actually do in fact exist. 
A low density tells us the system analysed is restrictive when actors have to establish 
relationships with other actors. In the other hand, in a high density system, relationships 
amongst actors can easily be made. 

3.3 Degree Centrality 

Degree is a measure that counts the number of ties an actor has. In the case where we are 
dealing with a network where direction of ties is important, we can distinguish between 
in-degree and out-degree. 

On one side, in-degree is the number of ties an actor receives. According to social 
network theory, if an actor receives many ties, it is often said he has high prestige 
because many other actors seek to direct him ties. This approach can be applied in our 
study because if an actor receives many ties, it means other actors trust him, so he has 
more prestige. He also has more power, because he can influence other actors as far as 
his opinions are taken into account. 

On the other hand, out-degree is the number of ties which begin with the actor himself. 
Actors with a high out-degree are able to make many others aware of their views. If an 
actor has a high out-degree it means he trusts a high number of other agents, and so he 
has more chances to ask for opinion/advice. We can say actors with high out-degree may 
be in advantaged positions because they have more alternative ways to satisfy needs and 
they are less dependent on other actors or they may have access to more resources. 



3.4 Network Centrality 

This measure is similar to the previous one, but here the whole network is analysed 
instead of each actor. There are several ways to calculate centrality in the network, and 
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each of them uses a different source that generates different rates. For example, it can be 
calculated by using degree centrality, closeness centrality, betweenness centrality and 
flow centrality. In our study we use the degree centrality calculated for each actor to 
calculate a value for the whole network. This decision has been taken because degree 
centrality is the only one which takes into account the immediate ties an actor has. The 
other ways to calculate centrality consider the indirect ties an actor has (i.e. actor 1 can 
reach actor 3 if there is an actor 2 connected to both of them). As we do not want to use 
this approach, we only use degree centrality. 

Using in and out-degrees, an index of network centrality can be calculated. First of 
all we need to define the star network. A star network is a network where there is one 
actor A connected to all the other actors in the network. The others have only one tie (a 
connection to A). The star network is the most centralised network for any number of 
actors. We can express the degree of variability in the degrees of actors in our analysed 
network as a percentage of that in a star network of same size. A different value for 
in-degree and out-degree is calculated. 

Another way to calculate centrality is by looking at the variation between the mean 
and the standard deviation for in and out-degrees. In a centralised network there is a high 
variation because there are huge differences within actors in and out-degrees, while in a 
network which is not centralised, variation tends to be lower. 

The network centralisation parameter gives us an idea of the amount of concentration 
or centralisation in the whole network. A high value means that the network is centralised, 
that is, there are several actors who have a high degree and several other actors who have 
a low degree (in or out). A low value means the network is not centralised, so the actors 
have a similar degree value. 

3.5 Clique Membership 

The next two measures are related to the substructures which may be present in the 
network. Divisions of actors into subgroups can be an important aspect of social structure, 
and it can be important in understanding how a network as a whole is likely to behave. 

The first structure we evaluate are cliques. A clique is a sub-set of a network in which 
actors are more closely tied to one another than to other members of the network. In real 
life people also tend to form cliques on the basis of age, gender, race and other criteria. 

The clique definition is very strong as a clique is a number of actors who have all 
possible ties present among themselves (i.e. in terms of graphs, a maximal complete 
subgraph). Three actors can easily form a clique, so we do not consider these kinds of 
structures in this measure. 

We calculate clique membership for each actor. First, all the cliques which are present 
in the network have to be found by considering only the ones with four or more actors. 
Then we get a clique membership which is the number of cliques on which an actor is a 
member. 

Clique membership gives us an idea about the tendency each actor has to form 
substructures in the graph. The fact that several actors have a high clique membership 
indicates that probably there are communities within the social network because these 
actors are highly related among themselves. If actors have a low clique membership it 
will be extremely difficult to find communities in the network. 
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3.6 Factions 

We have seen that cliques are very restrictive, because there must be all the possible ties 
present to form a clique. A less strict division would allow some ties between groups 
and also less than full density within them. So, the last measure we propose is factions. 
In network terms, it is possible to define partitions of the network by grouping together 
actors on the basis of the similarities in which they are tied. 

Using the power of computers it is possible to search for partitions of a network into 
groups that maximise the similarity of the patterns of connections of actors within each 
group. 

This method divides our network into the number of factions we want. The output is 
a set of different groups where actors are more likely to be tied to each other than with 
actors from other groups. This helps us to identify communities within our network. 



4 Running Example 

In an attempt to study the collaboration among users, our research group implemented 
GenialChef 1 , a restaurant recommender system developed within the IRES Project 2 . 
GenialChef is the basis of our experimentation. The details of this implementation are 
extensively explained in [9]. As a summary, GenialChef is a multi-agent system that rec- 
ommends interesting restaurants to its users. The agents making up this system can be 
grouped into service agents and personal agents (PA)(the system architecture is shown 
in Figure 1). The service agents provide objective information to the PAs: the restau- 
rant server agents (RSA) provide information about restaurants and the personal agent 
facilitator (PAFA) acts as a broker agent and is in charge of assisting the PAs in finding 
other PAs. PAs provide personalised information to their users. Every user has a PA in 
the system, which encapsulates his/her user profile and is in charge of recommending to 
him/her interesting restaurants. 

In order to take advantage of the collaborative world, PAs exchange information by 
means of two new information filtering methods: the opinion-based filtering method 
and the collaborative filtering method through trust. Their main idea is to consider other 
agents as personal entities on which you can either rely or not. Thus, PAs only collaborate 
with reliable agents. Reliability is expressed through a trust value with which each agent 
labels its neighbours [10]. Once the agent has a set of reliable agents, it can use them 
to filter information. When an agent is not sure about a recommendation or discovers 
a new item, it asks the reliable agents for their opinion and uses their trust values to 
decide whether the item is interesting for the user or not. Moreover, PAs can ask to their 
friends for advice, that is, ask about new items that could be of interest to the other 
user. We suppose that similar agents provide pertinent opinions, but they may also give 
inadequate ones. Therefore, trust should be modified depending on the results of the 
recommendations in order to improve acquaintances. 



1 GenialChef was awarded the prize for the best university project at the E-TECH 2003. 

2 The IRES Project was awarded the special prize at the AgentCities Agent Technology Compe- 
tition. 
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Fig. 1 . System Architecture 



Thus, after a period of time, each agent has a list of reliable agents with whom to 
collaborate in case of need (contact list). Clearly, this output can be viewed as a graph 
representing a social network. 

Therefore, each PA is represented by an actor of the graph, and the trust relationships 
each PA develops are directed ties among them. If a PA A trusts in another PA B, there 
would be a directed tie from A to B. One can find these relationships through the contact 
list of reliable agents that each PA has. In order to analyse the evolution of the system, 
two stages are considered. The first one is taken just when the system begins to execute, 
and the last one is taken at the end of the execution. For instance. Figure 2 and Figure 3 
show the layout of the social networks obtained in our experiments at the beginning 
and at the end of the execution respectively. Following the example, looking at Figure 3, 
salaJ2 trusts pages, llado and mirocoll opinions/advice and collaborates in cunatjnangui 
recommendations . 

It is also important to keep in mind that our social networks have locality 1 , which 
means that only the direct ties an actor has are taken into account. For example, having 
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Fig. 2. Network 1: Graph layout of the relations created among PAs at the beginning of the 
execution, presented as a social network using Netdraw visualization program 

three PAs A, B and C where A trusts in B and B trusts in C, does not necessarily mean 
that A trusts C. Therefore, some of standard techniques usually used in social network 
analysis can not be applied to our problem. 



5 Experimental Results 

We have used the measures proposed in Section 3 to analyse how the collaboration 
is done in a concrete framework. In particular, the collaboration among PAs from the 
recommender system explained in Section 4 has been evaluated. In order to do that, a 
simulator based on the “profile discovering procedure” has been developed [11], which 
allows us to perform thousands of repeatable and perfectly controlled experiments. 

In this section we analyse the simulation results of a 60 day long trial with 40 real user 
profiles extracted from our university staff. In particular we analyse the social networks 
evolution, taking one picture of the social network at the beginning of the simulation 
(just after the startup) and another one at the end of the simulation in order to analyse the 
evolution of the network during the experiment. It has been demonstrated that the use of 
collaboration increases the performance of the system in [9] . Moreover, the performance 
of the system has been studied using different parameters, so we know which parameters 
make the system perform better. Therefore, we can extract some conclusions about what 
the measures obtained in the analysis of the social network should be in order to get 
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Fig. 3. Network 2: Graph layout of the relationships created among PAs at the end of the 
execution 



a system with a higher performance. The simulation was performed with the optimal 
parameters studied in [9]. 

In order to evaluate the collaboration performed in this simulation, we have used 
UCINet [4], which is a software designed to represent and analyse social networks. In 
particular, we examined the different measures explained in Section 3: size, density, 
degree centrality, network centrality, clique membership and factions. 

The first two measures to analyse are size and density. As we used 40 user profiles to 
run the simulation, the size of both resulting networks is 40. The maximum number of 
ties we could have in a fully saturated network of size 40 is 1560. In Network 1 we only 
have 150 real ties while in Network 2 we have 170 real ties. Both values are low and 
so the resulting density is 9.6% for Network 1 and 10.9% for Network 2. The lowness 
of the measures points out that, in general, PAs do not have many friends. With regard 
to the evolution, the density has increased during the experiment (there are 20 ties more 
at the end of the execution), which means that collaboration has allowed the PAs of the 
system to make new and trusted friends during the execution. 

With regard to out and in-degrees, Figure 4 shows the degrees for each PA at the 
beginning of the execution, and Figure 5 shows the degrees after the execution. Looking 
at both figures, the highest in-degree is for marc, who appears on 12 other PA’s contact 
lists. Therefore, he is the most prestigious one because there are more PAs who trust 
him. The highest out-degrees are those of cuf i, del_acebo and j ordif . They have 
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Fig. 4. Freeman’s Degree at the beginning of the execution 



the highest number of PA’s on their contact lists to ask for opinions/advice. If we take a 
look at Figures 2 and 3, we see that all these PAs are drawn in a position which is quite 
central (in the centre of the layout). Another thing to pay attention to is the fact that there 
is a large number (13 at the beginning and 12 at the end) of PAs with an out-degree of 0. 
This means they do not have any PA on their contact lists and they cannot ask for advice 
in the case of needing it. There are also 6 PAs at the beginning and 7 at the end with an 
in-degree of 0. This means they do not have any PAs who trust them. This is due to the 
fact that the profiles associated with the concrete PA are different from all the others. In 
the real world, there are people who like restaurants that we would never go to. If we 
asked for advice from people with different tastes that differ from ours and we got a bad 
recommendation, we probably would not trust them anymore. 

By analysing the evolution of these degrees it is possible to have an idea of the effects 
of collaboration in the system, and which agents are the most collaborative. For example, 
cuf i trusted 12 other PAs at the beginning and, by the end, this number had increased 
to 16, so through collaboration a friendship had been developed with 4 new PAs. On the 
other hand, llado had 3 other PAs who trusted him at the beginning and 8 at the end, 
so this demonstrates that llado is likely to develop new ties with other PAs who start 
trusting him during the execution of the system. So what the analysis of degrees shows 
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Degree 



Fig. 5. Freeman’s Degree at the end of the execution 



is which are the most and the least collaborative agents. It has been demonstrated that 
a system where the agents have too many friends does not perform better, and neither 
does a system with a low number of friends. 

Having analysed degrees for individual PAs, we can now analyse the degrees in the 
whole network. Network centrality for out-degree is 37.54% at the beginning and 33.53% 
at the end and for in-degree is 16.5% at the beginning and 20.38% at the end. Therefore, 
these values are quite low, and this result would indicate that the network is not very 
centralised. However we have to be aware of the fact that network density is very low 
and there are very few ties in the network. As a consequence this result may be altered. 
Table 1 shows the values of means, standard deviations and the coefficien ts of variation 
for each situation. Clearly the population is more homogeneous with regard to in-degree, 
but the fact is that both values are high, so it can be concluded that structural positions are 
heterogeneous and that network centralisation is high in both networks. By comparing 
Network l’s results with Network 2’s ones, there is a reduction of the coefficients of 
variation in the system after the execution. As a consequence, a homogeneous system 
should perform better than a heterogeneous one. 

Now we analyse possible network substructures. Figure 6 shows the number of 
cliques of which each actor is a member. As there are several actors who have a high 
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Table 1 . Comparison of degree means, standard deviations and coefficients of variation for both 
networks 
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Fig. 6. Personal agents membership to cliques of size 4 or more 



clique membership we can say, with a reasonable certainty, that there exists at least 
one community in our network. There are 19 actors in Network 1 and 18 in Network 2 
who have a 0 clique membership. This indicates that there are some actors who clearly 
do not belong to any community. The evolution shows that clique memberships at the 
beginning are similar to the ones at the end. The most changeable PAs regarding clique 
membership are the ones mentioned in the in and out-degree analysis, because the ones 
with the higher degree changes are the ones with the higher clique-membership changes. 

In order to corroborate the hypothesis we came up with when analysing clique mem- 
bership, network substructures are examined by means of factions. This analysis has 
only been made in Network 2 because the most relevant one is the resulting network at 
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the end of the execution. UCINet’s output shows that the best partition dividing the PAs 
in two factions is: 

Faction 1 moises, bosch, mangui, robert, munoz, llado, neret, israel, tomds, marc, santi, 
davidJ2, raiil, colomer, maki, teixidor, mirocoll 2, rafa, cufi, jordifi delMcebo 
Faction 2 betty, bianca, iriana, monti, poiis, pages, alicia, figui, matabosch, eduard, 
viceng, arnau, buixd, germana -mangui, marti, toni, lluisdZ, cunatjnangui, salaJ2 

We can see the distribution of ties inside and outside of th e factions created in Figure 
5. We can calculate the density of ties in the four different areas of Figure 7 in order to 
evaluate them. In region 1-1 the density is 0.28, in region 2-2 it is 0.06, and in regions 
1-2 and 2-1 the densities are 0.05 and 0.04 respectively. On one hand there is region 
1-1 (faction 7) with a huge concentration of ties compared to the others. This faction 
creates a community, because its PAs have developed a large number of relationships 
among themselves and as a consequence, they are the ones who collaborate the most. 
On the other hand, faction 2 has a low density. In fact, regions 1-2 and 2-1 have almost 
the same density as region 2-2 (faction 2). Therefore, faction 2 cannot be considered a 
community. This is the consequence of not having enough PAs in the network to form 
other communities. 



6 Related Work 

Social network analysis has been largely applied to other domains with different pur- 
poses. There has been a great deal of work on studying the relationships among Internet 
users [19]. The main objective of these studies is to find similar users on the Internet that 
could give useful information to others. For example, some papers use social networks 
in order to find communities of similar users from the Web [5, 1] or e-mail [8]. 

Several research groups have used social networks to study trust and reputation 
mechanisms in multi-agent systems where agents act as assistants for the members of 
an electronic community. For example, some papers address the problem of calculating 
the degree of agent reputation needed in order to collaborate in a multi-agent system 
[13, 15]. Others use social networks in e-commerce to support reputations for both 
expertise (providing good service) and helpfulness (providing good referrals) [17]. 

No similar work on the utilisation of social network analysis in order to evaluate how 
collaboration is done has been found. 



7 Conclusions and Further Work 

This paper is a first attempt to analyse how users/agents collaborate in a collaborative 
recommender system. Up to now, efforts in research have been directed towards develop- 
ing recommender systems with collaboration and demonstrating that their performance 
is better than the ones who do not use collaboration. The main objectives of these col- 
laborating systems have always been focused on finding just who the best candidates to 
collaborate with are. However, we do not know much about how the ideal collaboration 
model should be in order to optimise the performance of these systems. 
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Fig. 7. Factions tie representation 



Our proposal in order to evaluate how collaboration is done, is by the use of social 
networks as a tool to represent and analyse collaboration in recommender systems. In 
particular we propose some measures based on social network analysis that help us to 
understand general aspects about the composition of the collaborative network. Thus, 
measures such as size, density, degree centrality, network centrality, clique membership 
and factions help us to achieve our objective. 

Having made these measures, we used them to show how an analysis of a real 
system should be. In the analysed example, we were able to observe that at the end of 
the execution, the level of collaboration was quite low (although it was higher than at 
the beginning), that there was a homogeneous group of users who formed a community, 
and that there were several other users who were very isolated from the rest. 

The results presented in this paper have been obtained through an execution of our 
simulator using the parameters which have been demonstrated to be optimal in [9]. 

Thus, the next step in our work is to perform large-scale experiments with different 
parameters analysing in each of them the proposed measures. In doing so, we will be 
able to find out the impact of the parameters of our recommender system in the social 
network. In particular, we want to study how the best collaboration is obtained, for 
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example, in a centralised/decentralised network, in a dense/non-dense network or in a 
network with more/less communities. 

Moreover, we also want to analyse how social networks evolve over time. In this paper 
we have only considered two snapshots of the network: one at the beginning and the 
other one at the end of the execution. In the future we want to study the whole evolution, 
and so how relationships among users are generated and how they are dropped, the 
social evolution of a certain user inside the social network and the progressive creation 
of communities and their evolution. 
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Abstract. Recommendation agents employ prediction algorithms to provide 
users with items that match their interests. In this paper, we describe and 
evaluate several prediction algorithms, some of which are novel in that they 
combine user-based and item-based similarity measures derived from either 
explicit or implicit ratings. We compare both statistical and decision-support 
accuracy metrics of the algorithms against different levels of data sparsity and 
different operational thresholds. The first metric evaluates the accuracy in terms 
of average absolute deviation, while the second evaluates how effectively 
predictions help users to select high-quality items. Our experimental results 
indicate better performance of item-based predictions derived from explicit 
ratings in relation to both metrics. Category-boosted predictions can lead to 
slightly better predictions when combined with explicit ratings, while implicit 
ratings (in the sense that we have defined them here) perform much worse than 
explicit ratings. 



1 Introduction 

Recommendation systems [1] have been a popular topic of research ever since the 
ubiquity of the web made it clear that people of hugely varying backgrounds would 
be able to access and query the same underlying data. The initial human-computer 
interaction challenge has been made even more challenging by the observation that 
customized services require sophisticated data structures and well thought-out 
architectures to be able to scale up to thousands of users and beyond. 

In recent years, recommendation agents are extensively adopted by both research 
and e-commerce recommendation systems in order to provide an intelligent 
mechanism to filter out the excess of information available and to provide customers 
with the prospect to effortlessly find out items that they will probably like according 
to their logged history of prior transactions. 

1.1 Background 

Recommendation agents need to employ efficient prediction algorithms so as to 
provide accurate recommendations to users. If we define a prediction to be a number 
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value that expresses the predicted likelihood that a user will “like” an item, then a 
recommendation is defined as the list of N items with respect to the top N predictions 
from the set of items available. Improved prediction algorithms indicate better 
recommendations. This explains the essentiality of exploring and comprehending the 
broad characteristics and potentials of prediction algorithms and the reason why this 
work concentrates on this research direction. 

There are generally two methods to formulate recommendations both depending 
on the type of items to be recommended, as well as, on the way that user models [2] 
are constructed. The two different approaches are content-based [3, 4] and 
collaborative filtering [1, 3, 5, 17], while additional hybrid techniques have been 
proposed as well [3]. 

Approaches of Recommendation Algorithms 

Recommendation algorithms are classified to content-based and collaborative 
filtering based, while hybrid techniques have been proposed, as well, that provide a 
combination of the two. 

Content Based Recommendation Algorithms 

Content-based algorithms are principally used when documents are to be 
recommended, such as web pages, publications, jokes or news. The agent maintains 
information about user preferences either by initial user input about his interests 
during the registration process or by rating documents. Recommendations are formed 
by taking into account the content of documents and by filtering in the ones that 
better match the user’s preferences and logged profile. 

Collaborative Filtering Based Recommendation Algorithms 

Collaborative-filtering algorithms aim to identify users that have relevant interests 
and preferences by calculating similarities and dissimilarities between user profiles 
[16]. The idea behind this method is that, it may be of benefit to one’s search for 
information to consult the behavior of other users who share the same or relevant 
interests and whose opinion can be trusted. 

Challenges of Recommendation Agents 

The challenges for recommendation algorithms expand to three key dimensions, 
identified as sparsity, scalability and cold-start. 

Sparsity 

Even users that are very active, result in rating just a few of the total number of items 
available in a database. As the majority of the recommendation algorithms are based 
on similarity measures computed over the co-rated set of items, large levels of 
sparsity can be detrimental to recommendation agents. In [14], Huang and al. propose 
to deal with sparsity problem by applying an associative retrieval framework and 
related spreading activation algorithms to explore transitive associations among 
consumers through their past transactions and feedback. 

Scalability 

Recommendation algorithms seem to be efficient in filtering in items that are 
interesting to users. However, they require computations that are very expensive and 
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grow non-linearly with the number of users and items in a database. Therefore, in 
order to bring recommendation algorithms successfully on the web, and succeed in 
providing recommendations with acceptable delay, sophisticated data structures and 
advanced, scalable architectures are required. In [15], Cosley and al. describe an open 
framework for practical testing of recommendation systems in an attempt to provide a 
standard, public testbed to evaluate recommendation algorithms in real-world 
conditions. 

Cold-Start 

An item cannot be recommended unless a user has rated it. This problem applies to new 
and obscure items and is particularly detrimental to users with eclectic taste [6, 7]. 

1.2 Contributions 

The primary contributions of this paper are: 

• The utilization of explicit ratings in an “implicit” sense so as to enrich the user’s 
model, without actually prompting users to express their preference to categories 

• The introduction of item- similarity and user-similarity measures based on 
implicit ratings 

• The introduction of item-based and category-boosted prediction algorithms based 
on a range of similarity measures 

• The empirical analysis and evaluation of presented prediction algorithms 

1.3 Organization 

Section 2 describes a set of similarity measures to compare the relevance between 
users or items. These measures derive either from explicit or implicit ratings. Section 
3 describes a set of existing and newly introduced prediction algorithms that integrate 
the similarity measures. Section 4 presents the experimental evaluation metrics that 
we employ in order to compare the algorithms and we discuss the results of our 
evaluation. Section 5 summarizes our contributions and draws directions for further 
research. 

2 Similarity Measures 

In this section, we present a set of similarity measures based on the Pearson 
correlation coefficient, a metric of relevance between two vectors [8]. When the 
values of these vectors are associated with a user’s model then we call it user-based 
similarity , whereas when they are associated with an item’s model then we call it 
item-based similarity. The similarity measure can be effectively used to balance the 
ratings significance in a prediction algorithm and therefore to improve accuracy. 

There are several similarity algorithms that have been used: cosine vector 
similarity, Pearson correlation, Spearman correlation, entropy-based uncertainty 
measure and mean-squared difference. In [9] Breese et al. suggest that Pearson 
correlation performs better than cosine vector similarity, while in [10] Herlock et al. 
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suggest that Pearson’s correlation performs better than Spearman’s correlation, 
entropy-based uncertainty and mean-squared difference for collaborative filtering. 
According to these remarks we decide on Pearson correlation to calculate item-based 
and user-based similarities taking advantage of both explicit and implicit ratings. 

An explicit rating identifies the preference of a user to a specific item. A user is 
prompted by the agent’s interface to provide ratings for items so as to improve his 
model. The more ratings the user provides, the more accurate the recommendations 
provided to him are. Ratings range from 1 to 10 with 1 expressing greatest aversion to 
the item and 10 expressing greatest liking to the item. Explicit ratings are logged by 
the system and form the user’s model. 

An implicit rating [11, 12] identifies the preference of a user to specific 
categories 1 . We use here the term “implicit” somewhat excessively so as to express 
that a user is never actually prompted to express his preference to categories. We take 
advantage of the fact that an item belongs to some categories and we develop a user 
model based on category preference. If the explicit rating of a user concerning a 
specific item belonging to a set of categories is considered “good” then his model is 
updated so as to include the preference and vice versa. A rating is considered as 
“good” when it is greater than or equal to a threshold. 

Before describing the algorithms we introduce some definitions that facilitate the 
explanation process. We define 

• A set of m users U={ u x : x=l,2,...,m] 

• A set of n items /= { i x : x= 1 ,2, . . . ,n } 

• A set of p categories C={c x : x=l,2,...,p] 

• A set of q explicit ratings R={r x :x:l,2,...,q A q<m*n] 

• A set of r implicit ratings R’={r s ’: x=l,2,...,t A t < m*p} 

• The explicit rating of a user U r with reference to an item i h as r j 

• The average explicit rating of a user U as r 

In the sequence, we define three matrices that derive from our data, the user-item 
matrix, the user-category matrix and the item-category bitmap matrix. 

• User-item matrix is a matrix of users against items that have as elements the 
explicit ratings of users to items. Some of the user-matrix cells are not filled, as 
there are items that are not rated by some users. 

• User-category matrix is a matrix of users against item categories that have as 
elements, values that show the number of times a user has rated positively or 
negatively for a category. For each category we keep two columns, one for 
positive ratings and one for negative ratings. 

• Item-categoiy bitmap matrix is a matrix of items against categories that have as 
elements the value 1 if the item belongs to the specific category and the value 0 
otherwise. 

Similarity is calculated over the parts of two vectors that derive from one of these 
matrices as it is depicted by the shadowed parts in figure 1 . 

1 Items in the database belong to categories. 
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User-item matrix User-category matrix Item-category Bitmap 

Fig. 1 . The different matrixes intended for similarity algorithms 

2.1 User-Based Similarity 
Based on Explicit Ratings 

If we define the subset of items that users U r and ll. have co-rated as 

x y 

l' = {/ : X = 1,2, ...,n An < n] , where ll is the total number of items in the 

database, then similarity between two users is defined as the Pearson correlation of 
their associated rows in the user-item matrix and is given by equation 1 . 



ri 




Based on Implicit Ratings 

The values of the user-category matrix elements are used to calculate the preference 
of the user W v G £/ to the category C . G C and the result is considered as an 

C x 

implicit rating r u G R to that category, where r , = — *10 and 

C x + c 

pos neg 

C , C are respectively the number of positive and negative ratings 2 that user U 

x pos X neg x 

has implicitly given to category X . The similarity between the two users is defined as 
the Pearson correlation of their implicit ratings to categories C € C and is given by 
equation 2, where p is the number of categories available. 

2 Ratings are considered as positive or negative when they are greater or lower than a threshold 
respectively. 
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2.2 Item-Based Similarity 
Based on Explicit Ratings 

If we define the subset of users that have rated both items i x and i as 

U' = [u x : X = 1,2 ffl'Affl' < m} , where m is the total number of users in 

database, then the similarity between the two items is defined as the Pearson 
correlation of their associated columns in the user-matrix and is given by equation 3. 



m 




Based on Item-Category Bitmap 

There is also an alternative way to calculate the correlation between two items taking 
into account the categories in which they belong. The similarity between the two 
items is defined as the Pearson correlation of their associated rows in the item- 
category bitmap matrix and is given by equation 4. 

p 

v xy = sim(i x ,i y )= ,,=1 — — ( 4 ) 

V h = i V h = i 

where p is the number of categories and v . is a value equal to 1 if the item 
X belongs to category h and equal to 0 otherwise. 

3 Prediction Algorithms 

Prediction algorithms [9] try to guess the rating that a user is going to provide for an 
item. We will refer to this user as active user U a and to this item as active item i . 

These algorithms take advantage of the logged history of ratings and of content 
associated with users and items in order to provide predictions. 




